Boxing game processing method, display control method, position detection method, cursor control method, energy consumption calculating method and exercise system

ABSTRACT

Glove type input articles  7 L and  7 R are imaged by an imaging unit  51  of a cartridge  3  in order to calculate the positions thereof. Virtual screens are prepared respectively for the input articles  7 L and  7 R. Each virtual screen is divided into an immovable area, a straight area and a cross area, and determines the areas in which the current positions of the input articles  7 L and  7 R are located in the virtual screens, the origins of which are located in past positions of the input articles  7 L and  7 R as determined twice before. If the current position is located in the straight area, a straight punch is displayed; if the current position is located in the cross area, a cross punch is displayed; and if the current position is located in the immovable area, globes are displayed in accordance with the positions of the input articles  7 L and  7 R. The left and right hands are distinguished on the basis of positions predicted from the past positions of the input articles  7 L and  7 R.

TECHNICAL FIELD

The present invention relates to a boxing game processing method making use of glove type input articles to be imaged by a stroboscope and the related arts.

BACKGROUND ART

The boxing game system disclosed by Japanese Patent Published Application No. Hei 2.004-49436 filed by the present applicant comprises a game unit and glove type input devices, such that a game player grips the main bodies of the input devices and swings them as if he is actually boxing. A piezoelectric buzzer is provided in the main body in order to detect the acceleration of the input device when it is swung. The acceleration information is output as an infrared light signal, which is received and decoded by an infrared light signal receiving decoding unit provided of the game unit. The acceleration information as decoded is received by a game processor of the game unit, which then calculates the force of the punch corresponding to the swinging of the input device. A damage value is determined by the force of the punch and used to recalculate the physical energy value of an opposing boxer displayed on a monitor. When the physical energy value is exhausted, the opposing boxer gets knocked down.

In this game system, since the positions of right and left punches are fixed on the monitor, the player can throw a punch at the opposing boxer, who is moving and having a guard, by swinging the input devices at an appropriate timing. Namely, what the player can control is only the timing of throwing a punch.

In the competition game system disclosed in Japanese Patent Published Application No. Hei 2003-79945, not only the acceleration of a glove type input device but also the relative position to a monitor are calculated. Accordingly, a player can control his position from which a punch is thrown in the monitor.

However, the input device of the boxing game system or competition game system as described above incorporate a sensor, a microcomputer, and other electronic circuits, and thereby it increases the production cost and can be the cause of trouble. In addition to this, since the weight tends to increase, the operability is not necessarily so good. This is particularly important because the player has to violently move the input devices in the air.

Also, in the case of the above boxing game system and so forth, only one type of punches is available which can be thrown (for example, only a straight punch).

Furthermore, in the case of the above boxing game system, the distinction between right and left is made on the basis of values set in particular ports provided in the right and left input devices. However, if MCUs are not provided in the input devices, the distinction between right and left cannot be made. In the case of the competition game system, when the glove type input device gripped by the left hand and the glove type input device gripped by the right hand are crossed to switch the relative positions thereof between left and right, the game unit recognizes the glove type input device gripped by the right hand as the glove type input device gripped by the left hand, and the glove type input device gripped by the left hand as the glove type input device gripped by the right hand. Namely, in such a case, it is impossible to distinguish between the right hand and the left hand.

SUMMARY OF THE INVENTION

Thus, it is an object of the present invention to provide a boxing game processing method and the related arts for making it possible to distinguish between left and right, and increase the types of punches which can be thrown while improving the reliability and manipulability, such that it is possible to enjoy a boxing game by swinging glove type input articles of simple design in the air.

In accordance with a first aspect of the present invention, boxing game processing method comprises: an illumination step of emitting infrared light in a predetermined cycle to illuminate a left-handed glove type input article and a right-handed glove type input article which are provided respectively with retroreflective surfaces; an image generation step of imaging the left-handed glove type input article and the right-handed glove type input article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a differential data generation step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a position calculation step of calculating positional information of the left-handed glove type input article and the right-handed glove type input article on the basis of the differential data; an area determination step of determining in which area the position of the left-handed glove type input article is located in a first virtual screen which is divided into a straight area, a cross area and an immovable area, wherein said position of the left-handed glove type input article is a relative position which is indicated by current positional information of the left-handed glove type input article and converted into a coordinate system, the origin of which is located in the position indicated by past positional information obtained by tracing back for a predetermined number of times; an area determination step of determining in which area the position of the right-handed glove type input article is located in a second virtual screen which is divided into a straight area, a cross area and an immovable area, wherein said position of the right-handed glove type input article is a relative position which is indicated by current positional information of the right-handed glove type input article and converted into a coordinate system, the origin of which is located in the position indicated by past positional information obtained by tracing back for a predetermined number of times; a display step of displaying a left glove image which represents the left-handed glove type input article and a right glove image which represents the right-handed glove type input article in accordance with the result of determination in said area determination steps for the left-handed glove type input article and the right-handed glove type input article, wherein the first virtual screen and the second virtual screen are provided as mirror images each other in the right and left direction, and wherein in said area determination step for the left-handed glove type input article, an image showing that a left straight punch is thrown is displayed as the left glove image when the relative position indicated by the current positional information of the left-handed glove type input article is located in the straight area which does not include the origin, an image showing that a left cross punch is thrown is displayed as the left glove image when the relative position indicated by the current positional information of the left-handed glove type input article is located in the cross area which does not include the origin, an image showing that no left punch is thrown is displayed as the left glove image when the relative position indicated by the current positional information of the left-handed glove type input article is located in the immovable area which includes the origin, in said area determination step for the right-handed glove type input article, an image showing that a right straight punch is thrown is displayed as the right glove image when the relative position indicated by the current positional information of the right-handed glove type input article is located in the straight area which does not include the origin, an image showing that a right cross punch is thrown is displayed as the right glove image when the relative position indicated by the current positional information of the right-handed glove type input article is located in the cross area which does not include the origin, an image showing that no right punch is thrown is displayed as the right glove image when the relative position indicated by the current positional information of the right-handed glove type input article is located in the immovable area which includes the origin.

In accordance with this configuration, the area determination of the current position of the glove type input article is performed in the coordinate system, the origin of which is located in the position indicated by past positional information obtained by tracing back for the predetermined number of times. In other words, the origin is always located at the position determined by tracing back for the predetermined number of times from the position to be currently determined, and thereby the motion determination is based on the relative position of the glove type input article. Because of this, even if there are disparities in the body height of the player and in the distance between the player 11 and an imaging device for performing the image generation step, it is possible to display a constant glove image.

In order to facilitate understanding of this feature, an area determination process which is performed on the basis of the absolute position of the glove type input article in the differential image will be considered. In this case, the differential image corresponds to the virtual screen. For example, when comparing a short player and a tall player playing with the glove type input article in the same posture, needless to say, there is a difference between the positions of the glove type input article gripped by the short and tall players in the differential image.

Accordingly, even if the short and tall players perform the similar action, the area where the glove type input article 7L of one is located may be different from the area where the glove type input article 7L of the other is located.

For example, while the glove type input article is located in the “straight area” of the virtual screen when a tall player such as an adult throws a straight punch, the glove type input article may be located in the “immovable area” of the virtual screen when a short player such as a child throws a straight punch. In such a case, although the similar action is taken, the glove image as displayed is different between a tall player and a short player. This shortcoming results also from the disparity in the distance between the imaging unit and the player. It is not desirable that, in spite of the similar action, a different globe image is displayed depending upon the body height of the player or the distance between the imaging device and the player. In accordance with the present invention, this shortcoming can be avoided.

Incidentally, it is not a realistic idea to provide different virtual screens, each of which is divided into a plurality of areas, respectively for different heights of players. Also, in the case of the present invention, there are the two virtual screens respectively for the two glove type input articles, while the straight area, the cross area and the immovable area are defined for each of the glove type input articles. Accordingly, a variety of glove images can be displayed respectively for the glove type input articles in response to motions.

In order to facilitate understanding of this feature, it is assumed that only one virtual screen is provided for the two glove type input articles. In such a case, a punch thrown with the left-handed glove type input article is either a straight punch or a left cross punch (a punch toward the right), and a punch thrown with the right-handed glove type input article is either a straight punch or a right cross punch (a punch toward the left).

Accordingly, the left-handed glove type input article when throwing a straight punch and the right-handed glove type input article when throwing a right cross punch can be located in the same area. Needless to say, the opposite is true. In such a case, in spite of the different types of motions for left and right, the glove image corresponding to the left-handed glove type input article and the glove image corresponding to the right-handed glove type input article become similar, so that the glove image as displayed may not correspond to the actual motion by the player. For example, in the case where the left-handed glove type input article when throwing a straight punch and the right-handed glove type input article when throwing a right cross punch are located in the same “straight area” of the virtual screen, the same image of a straight punch is displayed and therefore it is not appropriate as the glove image corresponding to the right-handed glove type input article.

In this situation, eventually, glove images must be provided with no distinction between the types of punches with the glove type input articles. Accordingly, it means nothing if the “straight area” and the “cross area” are distinctively defined. In other words, the respective motions of the left-handed glove type input article and the right-handed glove type input article cannot be reflected in the glove images. In this regard, in accordance with the present invention, it is possible to display a variety of glove images (the animations of a straight punch and a cross punch) reflecting the motions of the glove type input articles respectively.

Furthermore, in accordance with the present invention, it is possible to display glove images reflecting the intention of the player. This point will be explained in detail. In accordance with the present invention, the glove image is displayed depending upon the area in which the current position is located in the coordinate system, the origin of which is located in the position obtained by tracing back for a predetermined number of times. In this case, if the current position is located in the “immovable area” including the origin, the image as displayed is indicative of the figure in which no punch is thrown. Accordingly, when the motion of the glove type input article is small, the current position is often located in the “immovable area”, and thereby it is avoided as much as possible to determine, as a punch, a small motion of the player which is not intended as a punch.

The boxing game processing method as recited in the above further comprises: a step of obtaining a first extraction point indicative of the position of the left-handed glove type input article or the left-handed glove type input article on the basis of the differential data; a step of predicting the current position of the left-handed glove type input article on the basis of past positional information of the left-handed glove type input article; a step of predicting the current position of the right-handed glove type input article on the basis of past positional information of the right-handed glove type input article; a step of calculating a first distance which is a distance between the first extraction point and the current position as predicted of the left-handed glove type input article; a step of calculating a second distance which is a distance between the first extraction point and the current position as predicted of the right-handed glove type input article; a step of setting the current position of the right-handed glove type input article to the first extraction point if the first distance is larger than the second distance, and setting the current position of the left-handed glove type input article to the first extraction point if the second distance is larger than the first distance; a step of calculating a third distance which is a distance between the second extraction point and the current position as predicted of the left-handed glove type input article; a step of calculating a fourth distance which is a distance between the second extraction point and the current position as predicted of the right-handed glove type input article; a step of setting the current position of the right-handed glove type input article to the second extraction point if the third distance is larger than the fourth distance, and setting the current position of the left-handed glove type input article to the second extraction point if the fourth distance is larger than the third distance.

Furthermore, since the current positions of the left-handed glove type input article and the right-handed glove type input article are determined on the basis of the currently predicted positions of the left-handed glove type input article and the right-handed glove type input article, even when the player moves such that the left-handed glove type input article and the right-handed glove type input article are crossed to switch the relative positions thereof between left and right, the positions thereof can be determined correctly as much as possible (that is, left and right can be distinguished from each other).

The boxing game processing method as recited in the above further comprises: a step of obtaining a maximum horizontal coordinate of pixels that have luminance values larger than a predetermined threshold value in an image on the basis of the differential data; a step of obtaining a minimum horizontal coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a maximum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a minimum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data, wherein said step of obtaining the first extraction point comprising: a step of obtaining a first horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a second horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the first horizontal distance is larger than the second horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the second horizontal distance is larger than the first horizontal distance; said step of obtaining the second extraction point comprising: a step of obtaining a third horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a fourth horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the third horizontal distance is larger than the fourth horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the fourth horizontal distance is larger than the third horizontal distance.

In accordance with this configuration, since two points are extracted (i.e., the coordinates of the first and second extraction points are determined) on the assumption that both the left-handed glove type input article and the right-handed glove type input article are imaged, it is possible to simplify the calculation for extracting the two points. This point will be explained in detail. If it is not assumed that both the two glove type input articles are imaged, one shape or two shapes must be detected in the differential image. This is because it is possible both that both the two glove type input articles are imaged and that only one article is imaged. Furthermore, it is required to calculate the center coordinates of one shape or two shapes as detected. Particularly, in the case where two shapes are located close to each other, it is difficult to determine which one or two glove type input article is imaged, and thereby the calculation of the center coordinates becomes quite difficult. In accordance with the present invention, since it is not necessary to perform the detection of the respective shapes and the calculation of the center coordinates, the above difficulties shall not rise and the calculation amount is small.

The boxing game processing method as recited in the above further comprises: a step of moving a cursor on a screen to follow the variation of the position of the left-handed glove type input article and/or the right-handed glove type input article; a step of displaying an input area on the screen for receiving an input from an operator; a step of moving the position of the cursor to a predetermined position in the input area if the cursor is located in a predetermined area including the input area irrespective of the positions of the left-handed glove type input article and the right-handed glove type input article; a step of displaying an image indicative of the elapsed time after the cursor is located in the predetermined position and/or the remaining time until a predetermined time elapses on the screen; and a step of performing a predetermined process when the cursor is located in the predetermined area at least in the predetermined time.

In accordance with this configuration, when the cursor is located in the predetermined area including the input area, the cursor is moved to the predetermined position in the input area irrespective of the positions of the glove type input articles, so that the player can easily move the cursor to the input area only by bring the cursor close to the input area. In other words, when the cursor is moved close to the input area, it is predicted that the player intends to move the cursor to the input area, and thereby the cursor is automatically moved to the input area for the purpose of lessening the operational burden of the player. In addition to this, since the elapsed time after the cursor reaches the input area and/or the remaining time until a predetermined time elapses are displayed, the player can easily know the remaining time until the predetermined time at which the predetermined process is performed, and thereby the user-friendliness for the player can be improved.

In accordance with a second aspect of the present invention, a display control method comprises: an illumination step of emitting infrared light in a predetermined cycle to illuminate a plurality of input articles which are provided respectively with retroreflective portions; an image generation step of imaging the plurality of input articles both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a differential data generation step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; and a position calculation step of calculating positional information of the plurality of input articles respectively in the image on the basis of the differential data, wherein a plurality of virtual screens are provided respectively corresponding to the plurality of input articles, said display control method further comprises: an area determination step of determining in which area the position of the input article is located in the corresponding virtual screen which is divided into a plurality of areas, wherein said position of the input article is a relative position which is indicated by current positional information of the input article and converted into a corresponding coordinate system, the origin of which is located, in the position indicated by past positional information of the input article obtained by tracing back for a predetermined number of times, wherein an image corresponding to each of the plurality of input articles is displayed in accordance with the result of area determination of the each of the plurality of input articles by said area determination step.

In accordance with this configuration, even if there are disparities in the body height of the player and in the distance between the player and an imaging device for performing the image generation step, it is possible to display a constant image corresponding to the input article.

This is the same as in the boxing game processing method in accordance with the first aspect. Also, in the case of the present invention, since the plurality of virtual screens are provided respectively corresponding to the plurality of input articles, the plurality of areas can be defined in the virtual screen separately for each of the input articles. Accordingly, a variety of glove images can be displayed respectively for the input articles in response to the motions.

This is also the same as in the boxing game processing method in accordance with the first aspect. In the display control method as described above, the predetermined number of times is a plural number.

In accordance with this configuration, in comparison with the case where the predetermined number of times is one, the displacement of the input article for a longer period can be used for determining the area, and thereby when the input article is continuously moved, appropriate area determination is possible along the motion thereof. Also, it is possible to enhance the difference between a small motion and a large motion of the input article.

In the display control method as recited in the above, the virtual screen is divided into at least two areas including a first area and a second area, and wherein in said area determination step, an image is displayed as an image corresponding to the input article for showing that an input is made when the relative position indicated by the current positional information of the input article is located in the first area which does not include the origin, and an image is displayed as an image corresponding to the input article for showing that no input is made when the relative position indicated by the current positional information of the input article is located in the second area which includes the origin.

In accordance with this configuration, it is possible to display images of the input articles reflecting the intention of the player. In other words, when the motion of the input article is small, the current position is often located in the second area, and thereby it is avoided as much as possible to determine, as an input, a small motion of the player which is not intended as an input.

This is also the same as in the boxing game processing method in accordance with the first aspect. In this description, “the image showing that no input is made” is an image showing a basic figure, and “the image showing that an input is made” is an image changing from the basic figure.

In the display control method as recited in the above, the virtual screen is divided into at least three areas including a first area, a second area and a third area, and wherein in said area determination step, an image is displayed as an image corresponding to the input article for showing that a first input is made when the relative position indicated by the current positional information of the input article is located in the first area which does not include the origin, an image which is different from the image showing that the first input is made is displayed as an image corresponding to the input article for showing that a second input is made when the relative position indicated by the current positional information of the input article is located in the second area which does not include the origin, and an image is displayed as an image corresponding to the input article for showing that no input is made when the relative position indicated by the current positional information of the input article is located in the third area which includes the origin.

In accordance with this configuration, since the virtual screen is divided into at least three areas for the area determination process, a variety of images can be displayed for the input article in accordance with the current position of the input article. Also, as has been discussed above, it is avoided as much as possible to determine, as an input, a small motion of the player which is not intended as an input. The meanings of “the image showing that no input is made” and “the image showing that an input is made” are the same as explained above.

In accordance with a third aspect of the present invention, position detection method comprises: a step of emitting infrared light in a predetermined cycle to illuminate a first input article and a second input article which are provided respectively with retroreflective portions a step of imaging the first input article and the second input article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a step of obtaining a first extraction point indicative of the position of the first input article or the second input article on the basis of the differential data; a step of obtaining a second extraction point indicative of the position of the first input article or the second input article on the basis of the differential data; a step of predicting the current position of the first input article on the basis of past positional information of the first input article; a step of predicting the current position of the second input article on the basis of past positional information of the second input article; a step of calculating a first distance which is a distance between the first extraction point and the current position as predicted of the first input article; a step of calculating a second distance which is a distance between the first extraction point and the current position as predicted of the second input article; a step of calculating a third distance which is a distance between the second extraction point and the current position as predicted of the first input article; a step of calculating a fourth distance which is a distance between the second extraction point and the current position as predicted of the second input article; a step of setting the current position of the second input article to the first extraction point if the first distance is larger than the second distance, and setting the current position of the first input article to the first extraction point if the second distance is larger than the first distance; a step of setting the current position of the second input article to the second extraction point if the third distance is larger than the fourth distance, and setting the current position of the first input article to the second extraction point if the fourth distance is larger than the third distance.

Furthermore, since the current positions of the first and second input articles are determined on the basis of the currently predicted positions of the first and second input articles, even when the player moves such that the first input article and the second input article are crossed to switch the relative positions thereof between left and right, the positions thereof can be determined correctly as much as possible.

This is also the same as in the boxing game processing method in accordance with the first aspect. The position detection method as recited in the above further comprises: a step of obtaining a maximum horizontal coordinate of pixels that have luminance values larger than a predetermined threshold value in an image on the basis of the differential data; a step of obtaining a minimum horizontal coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a maximum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a minimum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data, wherein said step of obtaining the first extraction point comprises: a step of obtaining a first horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a second horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the first horizontal distance is larger than the second horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the second horizontal distance is larger than the first horizontal distance; said step of obtaining the second extraction point comprises: a step of obtaining a third horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a fourth horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the third horizontal distance is larger than the fourth horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the fourth horizontal distance is larger than the third horizontal distance.

In accordance with this configuration, since two points are determined (i.e., the coordinates of the first and second extraction points are determined) on the assumption that both two the input articles are imaged, it is possible to simplify the calculation for extracting the two points. This is also the same as in the boxing game processing method in accordance with the first aspect. In accordance with a fourth aspect of the present invention, a cursor control method comprises: a step of emitting infrared light in a predetermined cycle to illuminate an input article which is provided with a retroreflective portion; a step of imaging the input article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a step of calculating the position of the input article on the basis of the differential data; a step of moving a cursor on a screen to follow the variation of the position of the input article; a step of displaying an input area on the screen for receiving an input from an operator; a step of moving the position of the cursor to a predetermined position in the input area if the cursor is located in a predetermined area including the input area irrespective of the position of the input article; a step of displaying an image indicative of the elapsed time after the cursor is located in the predetermined position and/or the remaining time until a predetermined time elapses on the screen; and a step of performing a predetermined process when the cursor is located in the input area at least for the predetermined time.

In accordance with this configuration, when the cursor is located in the predetermined area including the input area, the cursor is moved to the predetermined position in the input area irrespective of the positions of the input articles, so that the player can easily move the cursor to the input area only by bring the cursor close to the input area. In addition to this, since the elapsed time after the cursor reaches the input area and/or the remaining time until a predetermined time elapses are displayed, the player can easily know the remaining time until the predetermined time at which the predetermined process is performed, and thereby the user-friendliness for the player can be improved.

This is also the same as in the boxing game processing method in accordance with the first aspect. In accordance with a fifth aspect of the present invention, an energy consumption calculating method comprises: a step of emitting infrared light in a predetermined cycle to illuminate an operation article operated by a user; a step of imaging the operation article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a step of calculating state information of the operation article the basis of the differential data; and a step of calculating energy consumption when the user operates the operation article on the basis of the state information.

As described above, the energy consumption of the player can be easily calculated by the use of the result of stroboscopic imaging.

In the energy consumption calculating method as recited in the above, the state information is one of or any combination of two or more of speed information, moving direction information, moving distance information, velocity vector information, acceleration information, movement locus information, area information, and positional information.

In accordance with a sixth aspect of the present invention, an exercise system comprises: an infrared light emission unit operable to periodically emit infrared light to a retroreflective portion which an exerciser puts on; an infrared light image sensor operable to detect the infrared light as reflected by the retroreflective portion to obtain a series of image data a signal processing unit connected to said infrared light image sensor, and operable to generate a first image indicative of an exercise that the exerciser to do, receive the series of image data of the retroreflective portion from said infrared light image sensor while the exerciser does the exercise, calculate calorie consumption estimated of the exerciser, and generate a second image indicative of the calorie consumption in numbers, wherein the calorie consumption is calculated on the basis of the motion of the retroreflective portion corresponding to the exercise that the exerciser has done with reference to the series of image data obtained by said infrared light image sensor.

By this configuration, it is possible to effectively do an exercise while enjoying.

Also, the player is informed of the amount of exercise he actually do in terms of calorie consumption to maintain the health of body.

BRIEF DESCRIPTION OF DRAWINGS

The aforementioned and other features and objects of the present invention and the manner of attaining them will become more apparent and the invention itself will be best understood by reference to the following description of a preferred embodiment taken in conjunction with the accompanying drawings, wherein.

FIG. 1 is a block diagram showing the entire configuration of a boxing game system in accordance with the embodiment 1 of the present invention.

FIG. 2A is a perspective view showing a glove type input article 7L as seen from the front right direction in accordance with the embodiment 1 of the present invention.

FIG. 2B is a perspective view showing a glove type input article 7L as seen from the front left direction in accordance with the embodiment 1 of the present invention.

FIG. 2C is a perspective view showing a glove type input article 7L as seen from the bottom left direction in accordance with the embodiment 1 of the present invention.

FIG. 3 is a perspective view showing an adapter 1 and a cartridge 3 of FIG. 1.

FIG. 4 is a perspective view showing the adapter 1 of FIG. 1 as seen from the back side.

FIG. 5 is a view showing the electric configuration of the adapter 1 of FIG. 1.

FIG. 6 is a schematic diagram showing the electric configuration of the cartridge 3 of FIG. 1.

FIG. 7 is a cross sectional view showing the cartridge 3 of FIG. 1.

FIG. 8A is a view showing an example of a game mode selection screen displayed on a television monitor 5 of FIG. 1.

FIG. 8B is a view for explaining the selection operation in the game mode selection screen.

FIG. 9 is an explanatory view for showing the determination operation in the game mode selection screen displayed on the television monitor 5 of FIG. 1.

FIG. 10 is a view showing an example of a game screen as displayed on the television monitor 5.

FIG. 11 is an explanatory view for showing the globe detection process by the use of the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 12 is an explanatory view for showing the right/left determination process by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 13A is a view for explaining the process of calculating velocity vectors by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 13B is a view for explaining the process of determining the globe motion by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 13C is a view for explaining the process of calculating velocity vectors by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 13D is a view for explaining the process of determining the globe motion by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 13E is a view for explaining the process of calculating velocity vectors by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 13F is a view for explaining the process of determining the globe motion by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 14 is a flowchart showing an example of the overall process flow by the high speed processor 91 incorporated in the cartridge 3 of FIG. 1.

FIG. 15 is a flowchart showing an example of the imaging process of step S2 of FIG. 14.

FIG. 16 is a flowchart showing an example of the globe detection process of step S3 of FIG. 14.

FIG. 17 is a flowchart showing an example of the process of detecting the left, right, upper and lower ends in step S32 of FIG. 16.

FIG. 18 is a flowchart showing an example of the process of determining two points in step S33 of FIG. 16.

FIG. 19 is a flowchart showing an example of the selection process in step S5 of FIG. 14.

FIG. 20 is a flowchart showing an example of the process flow during fighting in step S6 of FIG. 14.

FIG. 21 is a flowchart showing an example of the right/left determination process in step S120 of FIG. 20.

FIG. 22 is a flowchart showing an example of the process of determining the globe motion in step S121 of FIG. 20.

FIG. 23 is a flowchart showing an example of the process of updating the positions of the gloves of the player's boxer in step S122 of FIG. 20.

FIG. 24 is a flowchart showing an example of the calorie consumption calculation process in step S8 of FIG. 14.

FIG. 25 is a view showing an exemplary screen in which intermediate results is displayed on the basis of the processing result in step S9 of FIG. 14.

FIG. 26 is a view showing an exemplary screen in which the outcome of fight is displayed on the basis of the processing result in step S11 of FIG. 14.

FIG. 27 is a view showing an exemplary screen in which the total outcome is displayed after the outcome of fight is displayed in FIG. 26.

FIG. 28 is a view showing an exemplary screen in which comments are displayed after the total outcome is displayed in FIG. 27.

FIG. 29 is a view showing an example of an exercise screen displayed on the basis of the exercise process “A” performed by the boxing game system in accordance with an embodiment 2 of the present invention.

FIG. 30 is a view showing an example of an exercise screen displayed on the basis of the exercise process “B” performed by the boxing game system in accordance with the embodiment 2 of the present invention.

FIG. 31 is a view showing an example of an exercise screen displayed on the basis of the exercise process “C” performed by the boxing game system in accordance with the embodiment 2 of the present invention.

FIG. 32 is a view showing another example of the exercise screen of FIG. 31.

FIG. 33 is a view showing an example of an exercise screen displayed on the basis of the exercise process “D” performed by the boxing game system in accordance with the embodiment 2 of the present invention.

FIG. 34 is a schematic diagram showing the process transition among the routines performed by the boxing game system in accordance with the embodiment 2 of the present invention.

FIG. 35 is a view showing an example of the save slot selection screen displayed in step S501 of FIG. 34.

FIG. 36 is a view showing an example of the exercise selection screen displayed in step S513 of FIG. 34.

FIG. 37 is a view showing an example of the level selection screen displayed in step S514 of FIG. 34.

FIG. 38 is a view showing an example of the contents of saved data displayed in step S518 of FIG. 34.

FIG. 39 is a view showing another example of the contents of saved data displayed in step S518 of FIG. 34.

FIG. 40 is a view showing a further example of the contents of saved data displayed in step S518 of FIG. 34.

BEST MODE FOR CARRYING OUT THE INVENTION

In what follows, several embodiments of the present invention will be explained in conjunction with the accompanying drawings. Meanwhile, like references indicate the same or functionally similar elements throughout the respective drawings, and therefore redundant explanation is not repeated.

Embodiment 1

FIG. 1 is a block diagram showing the entire configuration of a boxing game system in accordance with the embodiment 1 of the present invention. As shown in FIG. 1, this boxing game system comprises an adapter 1, a cartridge 3, a glove type input article 7L (not shown in the figure), a glove type input article 7R and a television monitor 5. The adapter 1, the cartridge 3 and the glove type input articles 7L and 7R make up a boxing game system.

The cartridge 3 is inserted into the adapter 1. On the other hand, the adapter 1 is connected to the television monitor 5 through an AV cable 9. The glove type input article 7L and the glove type input article 7R are gripped by the left hand and the right hand of a player 11 respectively.

FIG. 2A is a perspective view showing the glove type input article 7L of FIG. 1 as seen from the front right direction; FIG. 2B is a perspective view showing the glove type input article 7L as seen from the front left direction; FIG. 2C is a perspective view showing the glove type input article 7R as seen from the bottom left direction. In this case, “front”, “left” and “right” indicate the directions as viewed from the player 11.

As shown in FIGS. 2B and 2C, retroreflective sheets 21 a and 21 b are attached respectively to the front bottom portion and the left side portion of the glove type input article 7L. Also, as shown in FIG. 2C, a grip 23L is fixed between the right side inner surface and the left side inner surface of the glove type input article 7L. The player 11 grips this grip 23L with the left hand. The glove type input article 7R to be gripped by the player 11 with the right hand is designed in the form of a mirror image of the glove type input article 7L.

In this case, the retroreflective sheets 21 a and 21 b are functionally equivalent to each other, and since they cannot be distinguished with the resolution of an image sensor 161 to be described below and used in the present embodiment, the retroreflective sheets 21 a and 21 b can be recognized as one retroreflective sheet. In other words, while these sheets are provided as separate parts, this configuration is not requisite but they can be integrated. Incidentally, the term “retroreflective sheet 21” is used to generally represent the retroreflective sheets 21 a and 21 b.

FIG. 3 is a perspective view showing the adapter 1 and the cartridge 3 of FIG. 1. FIG. 4 is a perspective view showing the adapter 1 as seen from the back side.

As shown in FIG. 3, the adapter 1 has a flat rectangular parallelepiped shape with an upper face, a lower face, a right and a left side face, a front and a back face. The adapter 1 is provided with a power supply, switch 45, a reset switch 43 and a power lamp 41 in the left hand side of the front face, and an infrared filter 33 in the right hand side of the front face. This infrared filter 33 is a filter capable of removing incident light except infrared light in order to only pass infrared light, and provided with an infrared sensor (not shown in the figure) located behind this infrared filter 33. In addition, arrow keys 37 a to 37 d are provided on the upper face of the adapter 1 in the vicinity of the front edge thereof. Furthermore, there are provided a cancel key 39 in the left hand side of the arrow key 37 a and an enter key 35 in the right hand side of the arrow key 37 d.

As shown in FIG. 4, an AV jack 83, a power jack 85, a video jack 81 V, an L channel audio jack 81L and an R channel audio jack 81R are provided in the back face of the adapter 1. Incidentally, the term “AV jack 81” is used to generally represent the video jack 81 V, the L channel audio jack 81L and the R channel audio jack 81R. The AV jack 83 is an external output terminal, and connected to an external input terminal of the television monitor 5. On the other hand, the AV jack 81 is an input terminal which can be connected to the output terminal of a variety of external equipments (for example, DVD (digital versatile disc) player).

An opening is formed in the middle position of the upper surface of the adapter 1 while a top plate 31 is disposed therein so that its upper face is approximately flush with the upper face of the adapter 1. Inside the adapter 1, there is an elevator mechanism which urges upward and supports the top plate 31 so that the upper face of the top plate 31 is located at the height as described above. The top plate 31 is supported to move up and down in the opening by this elevator mechanism. The cartridge 3 can be connected to a connector 32 by placing and pushing down the cartridge 3 on this top plate 31, and sliding the cartridge 3 toward the front face (refer to FIG. 1). This cartridge 3 contains a high speed processor 91, a memory 93 and the like to be described below. Also, needless to say, when the cartridge 3 is pushed down on the top plate 31, the downward movement distance of the top plate 31 is restricted by the elevator mechanism so that the cartridge 3 stops at a predetermined height.

Returning to FIG. 3, the cartridge 3 comprises a flat rectangular parallelepiped main body and an imaging unit 51. The front face of the main body of the cartridge 3 is provided with a connector section 57 having terminals t1 to t24 to be described below with which it is connected to the connector 32 of the adapter 1. The imaging unit 51 is mounted on the upper face of the main body of the cartridge 3. In this case, the imaging unit 51 is fitted in order that the surface thereof is inclined at a predetermined angle (for example, 40 degrees) relative to the surface of the cartridge 3. The imaging unit 51 is provided with a circular infrared filter 55 in the center portion of the surface thereof around which infrared light emitting diodes 53 a to 53 d are arranged. Meanwhile, the term “infrared light emitting diode 53” is used to generally represent each of the infrared light emitting diodes 53 a to 53 d.

FIG. 5 is a view showing the electric configuration of the adapter 1. As shown in FIG. 5, this adapter 1 includes the connector 32, an extension connector 63, an extension connector peripheral circuit 65, the reset switch 43, a crystal oscillator circuit 67, a key block 69, an infrared signal receiver circuit (IR receiver circuit) 71, an audio amplifier 73, an internal power supply voltage generation circuit 75, a power supply circuit 79 comprising an AC/DC converter and the like, the power supply switch 45, a switching regulator 77, the power jack 85, the AV jack 83, the video jack 81 V, the L channel audio jack 81L, and the R channel audio jack 81R. The connector 32 has 24 terminals T1 to T24 and is covered by a shield member 61 which is grounded. The terminals T1, T2, T22 and T24 of the connector 32 are grounded.

The AC voltage as supplied from a power cable (not shown in the figure) is supplied to the power supply circuit 79 through the power jack 85. The power supply circuit 79 converts the AC voltage as given to a DC voltage, which is then output to a line w20 as a power supply voltage Vcc0. When turned on, the power supply switch 45 connects the line w20 and a line w54 to supply the switching regulator 77 with the power supply voltage Vcc0, and gives the AV jack 83 a video signal “VD” from a line w9 and audio signals “AL2” and “AR2” from the lines w12 and w13 respectively through lines w14, w15 and w16. Accordingly, the video signal “VD” and the audio signals “AL2” and “AR2” are given to the television monitor 5 through the AV cable 9, while the television monitor 5 displays an image of the video signal “VD” with sounds of the audio signals “AL2” and “AR2” output from speakers (not shown in the figure).

On the other hand, when turned off, the power switch 45 connects lines w17, w18 and w19 to lines w14, w15 and w16. By this configuration, a video signal as input from the video jack 81 V, an L channel audio signal as input from the L channel audio jack 81L and an R channel audio signal as input from the R channel audio jack 81R are given to the AV jack 83. Accordingly, the video signal and the audio signals as input from the jacks 81 V, 81L and 81R are transferred to the television monitor 5 from the AV jack 83 through the AV cable 9. As thus described, when the power supply switch 45 is turned off, it is possible to output the video signal and the audio signals as input from an external device through the jacks 81 V, 81L and 81R to the television monitor 5.

The switching regulator 77 receives the power supply voltage Vcc0 from the power supply circuit 79 through the line w54 when the power supply switch 45 is turned on, and generates a ground potential GND and the power supply voltage Vcc1 on the lines w50 and w22 respectively. On the other hand, when the power supply switch 45 is turned off, the switching regulator 77 does not receive the power supply voltage Vcc0, and thereby it does not generate the power supply voltage Vcc1.

The internal power supply voltage generation circuit 75 generates power supply voltages Vcc2, Vcc3 and Vcc4 respectively on the lines w23, w24 and w25 from the ground potential GND and the power supply voltage Vcc1 as supplied from the switching regulator 77. The line w22 is connected to the terminals T7 and T8 of the connector 32; the line w23 is connected to the terminals T11 and T12 of the connector 32; the line w24 is connected to the terminals T15 and T16 of the connector 32; and the line w25 is connected to the terminals T18 and T19 of the connector 32. It is assumed that Vcc0>Vcc1>Vcc2>Vcc3>Vcc4. Incidentally, when the power supply switch 45 is turned off, the power supply voltage Vcc1 is not generated, and thereby the power supply voltages Vcc1, Vcc2, Vcc3 and Vcc4 are not supplied to the cartridge 3 through the connector 32.

The audio amplifier 73 amplifies the R channel audio signal “AR1” as input through the line w11 which is connected to the terminal T21 and the L channel audio signal “AL1” as input through the line w10 which is connected to the terminal T20, and outputs the R channel audio signal “AR2” and L channel audio signal “AL2” as amplified to the lines w13 and w12. The line w9 for inputting the video signal “VD” to the power supply switch 45 is connected to the terminal T23 of the connector 32.

The lines w9, w12 and w13 are covered by a cylindrical ferrite 87 in order not to radiate electromagnetic waves therefrom.

The IR receiver circuit 71 digital demodulates the digital modulated infrared signal as received, and outputs digital demodulated signal to the line w8. The line w8 is connected to the terminal T17 of the connector 32.

The key block 69 includes the cancel key 39, the arrow keys 37 a to 37 d and the enter key 35 and is provided with a shift register (not shown in the figure). This shift register serves to convert parallel signals which are input from the respective keys 39, 37 a to 37 d and 35 and a terminal TE7 described below, into serial signals, and output the serial signals to the line w3. This line w3 is connected to the terminal T6 of the connector 32. In addition, the key block 69 is given a clock signal through the line w5 which is connected to the terminal T10 and a control signal through the line w4 which is connected to the terminal T9.

The crystal oscillator circuit 67 oscillates a clock signal at a predetermined frequency (for example, 3.579545 MHz), and supplies the clock signal to the line w2. The line w2 is connected to the terminal T3 of the connector 32.

The reset switch 43 outputs a reset signal, which is used for resetting the system, to the line w1. The line w1 is connected to the terminal T4 of the connector 32.

The extension connector 63 is provided with first to ninth terminals (referred to as terminals TE1 to TE9 in the following description). The terminals TE2, TE4 and TE6 are connected to the terminals T13, T14 and T5 of the connector 32 respectively through the extension connector peripheral circuit 65. Accordingly, signals can be input from and output to the external device connected to the extension connector 63 through the terminals TE2, TE4 and TE6. The lines w4 and w5 are connected to the terminal TE 9 and TE 8 respectively. Accordingly, the external device connected to the extension connector 63 can receive the same clock signal through the terminal TE8 as the key block 69, and receive the same control signal as the key block 69 through the terminal TE9.

The terminals TE3 and TE5 are supplied respectively with the power supply voltages Vcc1 and Vcc2 through the extension connector peripheral circuit 65. Accordingly, the power supply voltages Vcc1 and Vcc2 can be supplied to the external device connected to the extension connector 63 through the terminals TE3 and TE5. The terminal TE 1 is grounded. The terminal TE7 is connected to a predetermined input terminal of the above shift register included in the key block 69 through the extension connector peripheral circuit 65.

FIG. 6 is a schematic diagram showing the electric configuration of the cartridge 3. As shown in FIG. 6, the cartridge 3 includes a high speed processor 91, a memory 93, the imaging unit 51, terminals t1 to t24, an address bus 95, a data bus 97, and an amplitude setting circuit 99. The amplitude setting circuit 99 includes the resistors 101 and 103.

The high speed processor 91 includes a reset input port /RESET for inputting a reset signal, a clock input port XT for inputting a clock signal “SCLK2”, an input/output ports (I/O ports) IO0 to IOn (“n” is a natural number, for example, n=24) for inputting/outputting data, analog input ports AIN0 to AINk (“k” is a natural number, for example, k=6) for inputting analog signals, audio output ports AL and AR for outputting audio signals “AL1” and “AR1”, a video output port VO for outputting a video signal “VD”, control signal output ports for outputting control signals (for example, a chip enable signal, an output enable signal, a write enable signal and so on), a data bus, and an address bus. The memory 93 includes an address bus, a data bus, and control signal input ports for inputting control signals (for example, a chip enable signal, an output enable signal, a write enable signal and so forth). The memory 93 may be, for example, a ROM (read only memory), a flash memory, or any appropriate memory.

The control signal output ports of the high speed processor 91 are connected to the control signal input ports of the memory 93. The address bus of the high speed processor 91 and the address bus of the memory 93 are connected to the address bus 95. The data bus of the high speed processor 91 and the data bus of the memory 93 are connected to the data bus 97. In this case, the control signal output ports of the high speed processor 91 include an OE output port for outputting an output enable signal, a CE output port for outputting a chip enable signal, a WE output port for outputting a write enable signal, and so forth. Also, the control signal input ports of the memory 93 include an OE input port connected to the OE output port of the high speed processor 91, a CE input port connected to the CE output port of the high speed processor 91, a WE input port connected to the WE output port of the high speed processor 91, and so forth.

When receiving the chip enable signal, the memory 93 responds to the chip enable signal as the destination thereof to output a data signal in accordance with an address signal and the output enable signal which are given substantially at the same time as the chip enable signal. The address signal is input to the memory 93 through the address bus 95 while the data signal is input to the high speed processor 91 through the data bus 97. Also, when receiving the chip enable signal, the memory 93 responds to the chip enable signal as the destination thereof to receive and write a data signal in accordance with an address signal and the write enable signal which are given substantially at the same time as the chip enable signal. The address signal is input to the memory 93 through the address bus 95 while the data signal is input to the memory 93 from the high speed processor 91 through the data bus 97.

When the cartridge 3 is installed into the adapter 1, the terminals t1 to t24 are connected to the terminals T1 to T24 of the connector 32 of the adapter 1 in a one-to-one correspondence. The terminals t1, t2, t22 and t24 are grounded. The terminal t3 is connected to the amplitude setting circuit 99. Namely, the resistor 101 of the amplitude setting circuit 99 is connected to the terminal t3 at one terminal thereof, and connected to the clock input port XT of the high speed processor 91 and one terminal of the resistor 103 at the other terminal thereof. The other terminal of the resistor 103 is grounded. Namely, the amplitude setting circuit 99 is a resistive potential divider.

The clock signal “SCLK1” generated by oscillation of the crystal oscillator circuit 67 of the adapter 1 is input through the terminal t3 to the amplitude setting circuit 99 which then generates a clock signal “SCLK2” having an amplitude smaller than the clock signal “SCLK1” and outputs the clock signal “SCLK2” to the clock input port XT. In other words, the amplitude of the clock signal “SCLK2” is set to a value which is determined by the ratio between the resistor 101 and the resistor 103.

The terminal t4 is connected to the reset input port /RESET of the high speed processor 91. Also, one terminal of the resistor 105 and one terminal of the capacitor 107 are connected to the line through which the reset input port /RESET is connected to the terminal t4. The other terminal of the resistor 105 is supplied with the power supply voltage Vcc3, and the other terminal of the capacitor 107 is grounded.

The terminals t5, t13 and t14 are connected respectively to the I/O ports 1012, 1013 and 1014 of the high speed processor 91. Accordingly, the high speed processor 91 can input signals to and output signals from an external device connected to the extension connector 63 of FIG. 5 through the terminals t5, t13 and t14.

The power supply voltage Vcc1 is supplied from the terminals t7 and t8. The power supply voltage Vcc2 is supplied from the terminals t11 and t12. The power supply voltage Vcc3 is supplied from the terminals t15 and t16. The power supply voltage Vcc4 is supplied from the terminals t18 and t19. The power supply voltage Vcc2 is supplied to the analog circuitry of the high speed processor 91 while the power supply voltage Vcc3 is supplied to the digital circuitry of the high speed processor 91.

The terminals t6, t9, t10 and t17 are connected respectively to the I/O ports IO15, IO16, IO17 and IO18 of the high speed processor 91. Accordingly, the high speed processor 91 can receive a signal output from the key block 69 through the terminal t6. Also, the high speed processor 91 can output a control signal to an external device connected to the extension connector 63 and the key block 69 through the terminal t9. Furthermore, the high speed processor 91 can supply a clock signal to an external device connected to the extension connector 63 and the key block 69 through the terminal t10. Still further, the high speed processor 91 can receive the output signal of the IR receiver circuit 71 through the terminal t17.

The terminals t20 and t21 are connected to the audio output ports AL and AR of the high speed processor 91. The terminal t23 is connected to the video output port VO of the high speed processor 91. Accordingly, the high speed processor 91 can output the audio signals “AL1” and “AR1” to the audio amplifier 73 of the adapter 1 through the terminals t20 and t21, and output the video signal “VD” to the power supply switch 45 of the adapter 1 through the terminal t23.

Incidentally, the cartridge 3 is provided with a shield member 113. By virtue of the shield member 113, electromagnetic waves can be prevented, as much as possible, from leaking from the high speed processor 91 and the like as external radiation.

The imaging unit 51 includes the infrared emitting diode 53, an image sensor 161, an LED drive circuit 92 and the infrared filter 55. The output terminal of the image sensor 161 is connected to the analog input AIN0 of high speed processor 91.

The image sensor 161 operates in response to the clock signal “SCLK1” from the terminal t3. The frame output flag signal “FS” and the image data output trigger signal “STR” as output from the image sensor 161 are given respectively to the I/O ports 109 and 1010 of the high speed processor 91. The signal “FS” takes a high level during an exposure period and a low level during the period for transferring pixel data. The high speed processor 91 receives the pixel data from the image sensor 161 in response to the rising edges of the signal “STR”.

The I/O ports IO0 to IO6 of the high speed processor 91 are connected respectively to the control terminals IP0 to IP6 of the image sensor 161. The high speed processor 91 gives the image sensor 161 commands through the I/O ports IO0 to IO6, and supplies the image sensor 161 with data to be set in control registers thereof through the I/O ports IO0 to IO6.

The high speed processor 91 outputs a clock signal “RCLK” to the image sensor 161 through the I/O port 107 for storing data in the control registers. Also, the high speed processor 91 outputs a reset signal to the image sensor 161 through the I/O port 108.

Furthermore, the high speed processor 91 outputs an LED control signal to the LED drive circuit 92 through the I/O port IO11. The LED drive circuit 92 drives the infrared light emitting diode 53 in accordance with the LED control signal and the signal “FS”. By this process, the infrared light emitting diode 53 repeats turning on and off and serves as a stroboscope.

Next, the internal configuration of the high speed processor 91 will be briefly explained. Although not shown in the figure, the high speed processor 91 includes a CPU (central processing unit), a graphics processor, a sound processor and a DMA controller and so forth, and in addition to this, includes an A/D converter for receiving analog signals, and an input/output control circuit for receiving input signals such as key manipulation signals and infrared signals and giving the output signals to external devices.

The CPU takes control of the entire system and performs various types of arithmetic operations in accordance with a program stored in the memory 93.

The graphics processor constructs graphics data on the basis of data stored in the memory 93, and outputs a video signal “VD” which is generated on the basis of the graphics data for displaying it on the television monitor 5.

The graphics processor constructs graphics data by the use of a background screen, sprites and a bitmap screen. The background screen which covers entirety of the screen of the television monitor 5 comprises a two-dimensional block array. And each block comprises of a rectangular set of pixels. There are a first background screen and a second background screen respectively prepared as the background screen for showing depths in the background. The sprite consists of a rectangular set of pixels which can be relocated in any position of the screen of the television monitor 5. The bitmap screen consists of a two-dimensional pixel array, the size and position of which as displayed can be freely designated.

In addition to this, the high speed processor 91 includes a pixel plotter which is not shown in the figure but can perform drawing operations with individual pixels. The sound processor converts data stored in the memory 93 into sound data, and generates and outputs the audio signals “AL1” and “AR1” on the basis of the sound data. The sound data is synthesized by pitch conversion and amplitude modulation of PCM (pulse code modulation) data serving as the base data of tone quality. For the amplitude modulation, an envelope control function for reproducing waveforms of a music instrument is provided in addition to a volume control function performed in response to an instruction of the CPU.

In addition to this, the high speed processor 91 is provided with an internal memory (not shown in the figure) which is used as a working area, a counter area, a register area, a temporary data area, a flag area and/or the like area.

FIG. 7 is a cross sectional view showing the cartridge 3 of FIG. 1. As shown in FIG. 7, a lens unit 164 is located in the rear side of the infrared filter 55 and fitted on the substrate 167. The lens unit 164 includes a unit base 159, a lens holder 151, a concave lens 153, and a convex lens 157. The concave lens 153 is attached to the lens holder 151 fixed to the unit base 159 in the side facing the infrared filter 55 in parallel with the image sensor 161 mounted on the board 167. Also, the convex lens 157 is attached to the lens holder 151 in the side facing the image sensor 161 in parallel with the image sensor 161. Furthermore, there is a cavity (optical path) 155 between the concave lens 153 and the convex lens 157. The infrared light as transmitted through the infrared filter 55 is detected by the image sensor 161 after passing through the concave lens 153, the cavity 155 and the convex lens 157.

Not shown in the figure, the infrared light emitting diodes 53 a and 53 d are fixed to the LED holding member 165 and inserted into the holes of the cylindrical sections 163 a and 163 d respectively. The holes of these cylindrical sections 163 a and 163 d are formed completely through the surface so that the emitting portions of the infrared light emitting diodes 53 a and 53 d are exposed to the surface of the imaging unit 51. This is true also for the infrared light emitting diodes 53 b and 53 c.

As shown in FIG. 7, a substrate 169 is fitted inside of the main body of the cartridge 3 for mounting the high speed processor 91, the memory 93 and so forth. The substrate 169 is rectangular shaped in the plane view and has the terminals t1 to t24 along the front edge thereof which is a part of the connector section 57. The substrate 169 is covered by the shield member 171. The shield member 113 of FIG. 6 is made up of the shield member 171 and a shield member which is additionally provided on the bottom surface inside of the main body of the cartridge 3.

Next, the general outline of the process by the boxing game system will be explained. With reference to FIG. 6, the infrared light emitting diode 53 is driven by the LED drive circuit 92 in order to intermittently emit infrared light. The infrared light as emitted then intermittently illuminates the retroreflective sheets 21 of the glove type input articles 7L and 7R which are gripped by the player 11. The image sensor 161 then images the retroreflective sheets 21 which are intermittently illuminated with infrared light. Accordingly, the image sensor 161 alternately outputs the image data of the retroreflective sheets 21 which are illuminated with infrared light and the image data of the retroreflective sheets 21 which are not illuminated with infrared light to the high speed processor 91. In the case of the present embodiment, an image sensor of 32 pixels×32 pixels is used as the image sensor 161. Accordingly, 32 pixels 32 pixels of pixel data (luminance data for each pixel) is output as image data from the image sensor 161. The high speed processor 91 calculates differential image data between the image data obtained when infrared light is emitted and the image data obtained when infrared light is not emitted. Then, the high speed processor 91 calculates the positional information (determines the positions) of the respective glove type input articles 7L and 7R on the basis of this differential image data. The high speed processor 91 displays a game mode selection screen and a game screen on the television monitor 5 by performing various processes to be described below on the basis of the positional information of the respective glove type input articles 7L and 7R as calculated.

FIG. 8A is a view showing an example of the game mode selection screen displayed on the television monitor 5 of FIG. 1. FIG. 8B is a view for explaining the selection operation in the game mode selection screen. As illustrated in FIG. 8A, when the cartridge 3 is inserted into the adapter 1 and then the power supply switch 45 of the adapter 1 is turned on, the game mode selection screen is displayed on the television monitor 5 by the high speed processor 91.

The game mode selection screen includes a game mode display section 200, selection buttons 203U and 203D, an OK button 207 and a cursor 201. The selection buttons 203U and 203D include indicators 202U and 202D which are arrow shaped respectively. The OK button 207 includes an circular indicator 209.

The high speed processor 91 synchronizes the motion of the cursor 201 with the motion of the glove type input articles 7L and/or 7R as imaged by the image sensor 161. Accordingly, the player 11 can manipulate the cursor 201 by moving the glove type input articles 7L and/or 7R. In this case, when only one of the glove type input articles 7L and 7R is imaged, the motion of the cursor 201 is synchronized with the motion of the glove type input article as imaged, but when both the glove type input articles 7L and 7R are imaged, the motion of the cursor 201 is synchronized with the motion of the center position of these glove type input articles.

As illustrated in FIG. 8B, when the cursor 201 enters a selection acceptable area 211 including the selection button 203D, the high speed processor 91 moves the cursor 201 to the center position of the selection button 203D irrespective of the motion of the glove type input articles 7L and 7R. For the sake of clarity in explanation, the selection acceptable area 211 is illustrated in the figure while it is not displayed on the television monitor 5 in practice.

After moving the cursor 201 to the center position of the selection button 203D, the high speed processor 91 gradually fills the indicator 202D with a predetermined color as time passes in order to indicate the elapsed time. The indicator 202D is completely filled with the predetermined color after a predetermined time elapses. The high speed processor 91 fixes the selection operation after the predetermined time elapses, and displays the next game mode selection screen (the game mode selection screen of FIG. 9) on the television monitor 5 in accordance with the direction of the arrow of the indicator 202D.

However, if the player 11 greatly moves the glove type input articles 7L and/or 7R to locate the cursor 201 out of the selection acceptable area 211 before the predetermined time elapses, the selection operation is not fixed such that the color of the indicator 202D is returned to the initial color.

The selection operation by the use of the selection button 203U is same as the selection operation by the use of the selection button 203D, and therefore no redundant description is repeated.

FIG. 9 is an explanatory view for showing the determination operation in the game mode selection screen. As illustrated in FIG. 9, when the selection operation is fixed in the condition shown in FIG. 8B, the high speed processor 91 displays the game mode selection screen on the television monitor 5 in which the selectable game mode is “arbitrary match”. The “arbitrary match” mode is a game mode in which the player 11 can arbitrarily select a boxer as his opponent.

When the cursor 201 enters a determination acceptable area 213 including the OK button 207, the high speed processor 91 moves the cursor 201 to the center position of the OK button 207 irrespective of the motion of the glove type input articles 7L and 7R. For the sake of clarity in explanation, the determination acceptable area 213 is illustrated in the figure while it is not actually displayed on the television monitor 5.

After moving the cursor 201 to the center position of the OK button 207, the high speed processor 91 gradually fills the indicator 209 with a predetermined color in the clockwise direction as time passes in order to indicate the elapsed time. The indicator 209 is completely filled with the predetermined color after a predetermined time elapses. The high speed processor 91 fixes the determination operation after the predetermined time elapses, and enters the game mode displayed in the game mode display section 200 (“arbitrary match” in the case of FIG. 9).

However, if the player 11 greatly moves the glove type input articles 7L and/or 7R to locate the cursor 201 out of the determination acceptable area 213 before the predetermined time elapses, the determination operation is not fixed such that the color of the indicator 209 is returned to the initial color.

FIG. 10 is a view showing an example of a game screen (tournament or arbitrary match) as displayed on the television monitor 5. As shown in FIG. 10, the game screen includes a CPU boxer 215 and globes 217L and 217R of the boxer who is controlled by the player 11 (referred to herein as “player's boxer”).

The high speed processor 91 controls the motion (including punches) of the CPU boxer 215 in accordance with the program stored in the memory 93. Also, the high speed processor 91 controls the motion of the glove 217L in accordance with the motion of the glove type input article 7L imaged by the image sensor 161, and controls the motion of the glove 217R in accordance with the motion of the glove type input article 7R imaged by the image sensor 161. Accordingly, the player 11 can avoid and defend himself against a punch of the CPU boxer 215 by moving the glove type input articles 7L and 7R.

The game screen further includes a physical indicator 221 a and a mental indicator 221 b of the CPU boxer 215 and a physical indicator 223 a and a mental indicator 223 b of the player's boxer.

The physical indicators 221 a and 223 a indicate the physical energy of the CPU boxer 215 and the physical energy of the player's boxer respectively, and each time either boxer takes a punch, his physical energy as indicated is decreased. In this case, the decreased amount of physical energy is determined in accordance with the force of the punch as hit. The mental indicators 221 b and 223 b indicate the mental energy of the CPU boxer 215 and the mental energy of the player's boxer respectively, and each time either boxer takes a punch, his mental energy as indicated is more greatly decreased than his physical energy. However, his mental energy is recovered at a predetermined speed up to the residual physical energy at a maximum. When the mental energy is exhausted to zero, the high speed processor 91 judges that the boxer falls down on the ground. Then, if the boxer stays on the ground for a predetermined period, the high speed processor 91 judges that the boxer gets knocked out.

The game screen further includes a round indication section 219 in which the remaining time of the current round is displayed.

Next, a globe detection process, a right/left determination process and a globe motion determination process will be explained with reference to drawings.

FIG. 11 is an explanatory view for showing the globe detection process by the use of the high speed processor 91. An image of 32×32 pixels is illustrated in FIG. 11 on the basis of the differential image data which is generated from the image data obtained when infrared light is emitted and the image data obtained when infrared light is not emitted. In the figure, each of the small unit squares represents one pixel. Also, the origin of the XY coordinates is located at the upper left vertex.

This image includes two areas 251 and 253 having large luminance values. The areas 251 and 253 represent the retroreflective sheet 21 of the glove type input article 7L and the retroreflective sheet 21 of the glove type input article 7R. However, at this time, it cannot be determined which area corresponds to which glove type input article.

The high speed processor 91 first scans the differential image data from X=0 to X=31 with Y=0 as a start point, then Y is incremented followed by scanning the differential image data from X=0 to X=31 again. This process is repeated until Y=31 in order to completely scan the differential image data of 32×32 pixels and determine the upper end position minY, the lower end position maxY, the left end position minX and the right end position maxX of the pixel data greater than a threshold value “ThL”.

Next, the high speed processor 91 scans the differential image data in the positive x-axis direction from the coordinates (minX, minY) as a start point, in order to calculate the distance “LT” between the start point and the pixel at which the luminance value first exceeds the threshold value “ThL”. Also, the high speed processor 91 scans the differential image data in the negative x-axis direction from the coordinates (maxX, minY) as a start point, in order to calculate the distance “RT” between the start point and the pixel at which the luminance value first exceeds the threshold value “ThL”. Furthermore, the high speed processor 91 scans the differential image data in the positive x-axis direction from the coordinates (minX, maxY) as a start point, in order to calculate the distance “LB” between the start point and the pixel at which the luminance value first exceeds the threshold value “ThL”. Still further, the high speed processor 91 scans the differential image data in the negative x-axis direction from the coordinates (maxX, maxY) as a start point, in order to calculate the distance “RB” between the start point and the pixel at which the luminance value first exceeds the threshold value “ThL”.

If the distances satisfy LT>RT, the high speed processor 91 sets a first extraction point to the coordinates (maxX, minY), and if the distances satisfy LT≦RT, the high speed processor 91 sets the first extraction point to the coordinates (minx, minY). Also, if the distances satisfy LB>RB, the high, speed processor 91 sets a second extraction point to the coordinates (maxX, maxY), and if the distances satisfy LB≦RB, the high speed processor 91 sets the second extraction point to the coordinates (minX, maxY).

FIG. 12 is an explanatory view for showing the right/left determination process by the high speed processor 91. FIG. 12 shows the position TPL2 of the glove type input article 7L as determined just before (one video frame before) and the position TPL1 of the glove type input article 7L as determined twice before (two video frames before), and the position TPR2 of the glove type input article 7R as determined just before (one video frame before) and the position TPR1 of the glove type input article 7R as determined twice before (two video frames before). The positions TPL1, TPL2, TPR1 and TPR2 are positions in the image on the basis of the differential image data.

The high speed processor 91 calculates a velocity vector VL which has its start point at the position TPL1 and its end point at the position TPL2. Then, the predicted position TPLp of the glove type input article 7L is obtained as the end point of the velocity vector VL having the position TPL2 as its start point. On the other hand, the high speed processor 91 calculates a velocity vector VR which has its start point at the position TPR1 and its end point at the position TPR2. Then, the predicted position TPRp of the glove type input article 7R is obtained as the end point of the velocity vector VR having the position TPR2 as its start point.

The high speed processor 91 obtains the distance LD1 between the first extraction point TPN1 and the predicted position TPLp, the distance RD1 between the first extraction point TPN1 and the predicted position TPRp, the distance LD2 between the second extraction point TPN2 and the predicted position TPLp, and the distance RD2 between the second extraction point TPN2 and the predicted position TPRp.

If the distances satisfy LD1>RD1, the high speed processor 91 sets the current position of the glove type input article 7R to the first extraction point TPN1, and if the distances satisfy LD1≦RD1, the high speed processor 91 sets the current position of the glove type input article 7L to the first extraction point TPN1. Also, if the distances satisfy LD2>RD2, the high speed processor 91 sets the current position of the glove type input article 7R to the second extraction point TPN2, and if the distances satisfy LD2≦RD2, the high speed processor 91 sets the current position of the glove type input article 7L to the second extraction point TPN2. Incidentally, in the case where the left and right predicted positions TPLp and TPRp cannot be calculated, for example, just after the game starts, the coordinates of the glove type input article 7L are set to the coordinates of one of the first extraction point TPN1 and the second extraction point TPN2 the X-coordinate of which is “minX”, and the coordinates of the glove type input article 7R are set to the coordinates of the other glove type input article the X-coordinate of which is “maxX”.

As has been discussed above, since the first extraction point TPN1 and the second extraction point TPN2 are associated respectively with left and right or right and left on the basis of the left and right predicted positions TPLp and TPRp, the high speed processor 91 can properly recognize the glove type input articles 7L and 7R in the image on the basis of the differential image data even if the relative positions of the glove type input articles 7L and 7R are switched (crossed) between left and right.

FIGS. 13A, 13C and 13E are explanatory views for showing the process of calculating velocity vectors by the high speed processor 91, and FIGS. 13B, 13D and 13F are explanatory views for showing the globe motion determination process by the high speed processor 91. These drawings are explanatory views for showing the motion determination process of the glove type input article 7L.

Positions TPL1 to TPL3 of the glove type input article 7L are illustrated in FIGS. 13A, 13C and 13E. The positions TPL1 to TPL3 are positions in the image on the basis of the differential image data. As shown in these drawings, the high speed processor 91 calculates a velocity vector “V” by the use of the position of the glove type input article 7L two video frames before (for example, the position TPL1) as a start point, and the current position (for example, the position TPL3) as an end point.

Then, the high speed processor 91 places the start point TPL1 of the velocity vector “V” at the origin in the virtual screen (32×32 pixels) as illustrated in FIGS. 13B, 13D and 13F, and determines in which area the end point TPL3 is located. If the end point TPL3 of the vector “V” is located in an area “immovable”, the high speed processor 91 determines that the player 11 does not throw a left punch (refer to FIG. 13B). In other words, although the glove type input article 7L is actually moved, the high speed processor 91 does not recognize it as a punch. If the end point TPL3 of the vector “V” is located in an area “straight”, the high speed processor 91 determines that the player 11 does throw a left straight punch (refer to FIG. 13D). In other words, the high speed processor 91 determines that the glove type input article 7L is moved straightly. If the end point TPL3 of the vector “V” is located in an area “cross”, the high speed processor 91 determines that the player 11 does throw a left cross punch (refer to FIG. 13F). In other words, the high speed processor 91 determines that the glove type input article 7L is moved in the form of a cross.

As understood from FIGS. 13B, 13D and 13F, in such a motion determination process, the processor 91 places the start point TPL1 of the velocity vector “V” at the origin of the virtual screen. The origin of the virtual screen is located at the center position of the lower side thereof.

For the glove type input article 7R, a virtual screen is provided by laterally inverting the virtual screen as illustrated in FIGS. 13B, 13D and 13F, and the motion (immovable, straight, cross) determination process is performed in the same manner as that for the glove type input article 7L.

Next, with reference to a flowchart, the process flow of the boxing game system will be explained.

FIG. 14 is a flowchart showing an example of the overall process flow by the high speed processor 91. As shown in FIG. 14, the high speed processor 91 performs an initialization process in step S1. More specifically speaking, the system hardware and the respective variables are initialized.

The high speed processor 91 performs the imaging process of the glove type input articles 7L and 7R in step S2. Also, in step S3, the high speed processor 9 performs the process of detecting the glove type input articles 7L and 7R on the basis of the result of the imaging process of in step S2. The high speed processor 91 proceeds to step S5 if the game state is “game mode selection”, proceeds to step S6 if the game state is “fighting”, proceeds to step S7 if the game state is “falling down”, proceeds to step S8 if the game state is “end of round”, and proceeds to step S10 if the game state is “end of bout”. However, the game mode is initialized to “game mode selection” at power up.

In step S5, the high speed processor 91 performs the game mode selection process in response to the motion of the glove type input articles 7L and 7R. In step S6, the high speed processor 91 controls the motion of the CPU boxer 215, and the motion of the gloves 217L and 217R in response to the motion of the glove type input articles 7L and 7R. The high speed processor 91 performs the fighting process between the CPU boxer 215 and the player's boxer in this way. In this case, the high speed processor 91 decreases the mental energy indicated by the mental indicator 221 b or 223 b of the boxer who takes a punch, judges falling down when the mental energy is exhausted to zero, and sets the game mode to “falling down”. Also, when a predetermined time elapses and one round ends, the high speed processor 91 sets the game mode to “end of round”. Furthermore, when the final round ends, the high speed processor 91 sets the game mode to “end of bout” after performing the process of steps S8 and S9.

In this example, if the end point of the velocity vector “V” of the glove type input article 7L is located in the straight area or the cross area shown in FIG. 13, the high speed processor 91 determines that a punch is thrown with the glove type input article 7L. This is true also for the glove type input article 7R.

In step S7, the high speed processor 91 performs the process for the falling down. In the case where the player's boxer falls down, if the glove type input articles 7L and 7R are swung for a predetermined number of times which is determined in accordance with the remaining physical energy before the count of 10, the game state is set to “fighting” by this process. If the glove type input articles 7L and 7R are not swung for the predetermined number of times before the count of 10, the high speed processor 91 judges a knockout and set the game state to “end of bout”.

On the other hand, when the CPU boxer falls down, the game state is set to “fighting” or “end of bout” in accordance with the behavioral algorithm of the CPU boxer.

In step S8, the high speed processor 91 calculates the calorie consumption of the exercising player 11 in the current round. In step S9, the high speed processor 91 performs the settings of animation and the display positions in order to display the calorie consumption of the player 11 and the points of the respective boxers in the round.

In step S10, the high speed processor 91 sums up the energy consumptions of the player 11 in the respective rounds to calculate the total calorie consumptions of the player 11 through the bout. In step S11, the high speed processor 91 performs the settings of animation and the display positions in order to display the total calorie consumption of the player 11 and the outcome of the bout. On the other hand, when the time runs out, the outcome of the bout is determined.

If there is an interrupt by a video system synchronous signal in step S12, the process proceeds to step S13, otherwise the process repeats the same step S12. The interrupt by a video system synchronous signal is issued at 1/60 second intervals.

In step S13, the high speed processor 91 updates the display image (video frame) of the television monitor 5 on the basis of the animation and display positions as set in steps S5 to S11.

The sound process in step S14 is performed when an audio interrupt is issued for outputting game music sounds, and other sound effects.

FIG. 15 is a flowchart showing an example of the imaging process of step S2 of FIG. 14. As shown in FIG. 15, the high speed processor 91 turns on the infrared light emitting diode 53 in step S20. In step S21, the high speed processor 91 acquires, from the image sensor 161, image data obtained when infrared light is emitted, and stores the image data in the internal memory.

As has been discussed above, the present embodiment makes use of the image sensor 161 of 32 pixels×32 pixels. Accordingly, 32 pixels 32 pixels of pixel data (luminance data for each pixel) is output as image data from the image sensor 161. This pixel data is converted into digital data by the A/D converter and stored in the internal memory as the array elements “P1 [X] [Y]”.

In step S22, the high speed processor 91 turns off the infrared light emitting diode 53. In step S23, the high speed processor 91 acquires, from the image sensor 161, image data (32 pixels 32 pixels of pixel data (luminance data for each pixel)) obtained when infrared light is not emitted, and stores the image data in the internal memory. In this case, the image data obtained when infrared light is not emitted is stored in the array elements “P2[X][Y]” of the internal memory.

The stroboscope imaging is performed in this way. Since the image sensor 19 of 32 pixels×32 pixels is used in the case of the present embodiment, X=0 to 31 and Y=0 to 31.

FIG. 16 is a flowchart showing an example of the globe detection process of step S3 of FIG. 14. As shown in FIG. 16, in step S30, the high speed processor 91 calculates the differential data between the pixel data “P1 [X] [Y]” acquired when infrared light is emitted and the pixel data “P2[X] [Y]” acquired when infrared light is not emitted, and the differential data is assigned to the respective array elements “Dif [X] [Y]”. In step S31, when the process of calculating the 32×32 pixels of the differential image is finished, the high speed processor 91 proceeds to step S32, but if the process is not finished yet, the high speed processor 91 proceeds to step S30. In this way, the high speed processor 91 repeats the process of step S30 to generate differential image data between the image data obtained with infrared light illumination and the image data obtained without infrared light illumination. The noise due to lights other than reflected light from the retroreflective sheets 21 of the glove type input articles 7L and 7R can be eliminated, as much as possible, by obtaining differential image data (differential image), and thereby the glove type input articles 7L and 7R can be detected with a high degree of accuracy.

In step S32, the high speed processor 91 performs the process of detecting the left, right, upper and lower ends (minX, maxX, minY, maxY) as explained with reference to FIG. 11. In step S33, the high speed processor 91 performs the process of determining the positions of two points (the first extraction point (Xtp[0], Ytp[0]) and the second extraction point (Xtp[1], Ytp[1])) as explained with reference to FIG. 11. In step S34, the high speed processor 91 calculates the center coordinates between the first extraction point (Xtp[0], Ytp[0]) and the second extraction point (Xtp[1], Ytp[1]). Then, the center coordinates are converted into the corresponding screen coordinates.

FIG. 17 is a flowchart showing an example of the process of detecting the left, right, upper and lower ends in step S32 of FIG. 16. This flowchart is an example of the process of detecting the left, right, upper and lower ends as explained with reference to FIG. 11.

As shown in FIG. 17, the high speed processor 91 assigns “0” to “X”, “Y”, “maxX”, “maxY” and “k” in step S40. Also, the high speed processor 91 assigns “31” to “minX” and “minY”.

In step S41, the high speed processor 91 compares the array element “Dif [X] [Y]” with a predetermined threshold value “ThL”. If the array element “Dif [X] [Y]” is larger than the predetermined threshold value “ThL” in step S42, the high speed processor 91 proceeds to step S43, and conversely if the array element “Dif [X] [Y]” is not larger than the predetermined threshold value “ThL”, the high speed processor 23 proceeds to step S55.

The process in steps S41 and S42 is the process for detecting whether or not the glove type input articles 7L or 7R is imaged. Since the glove type input articles 7L and 7R are provided with the retroreflective sheets 21, when the glove type input article 7L or 7R is imaged, the luminance values of the pixels on the differential image corresponding to the retroreflective sheet 21 become large. Because of this, the pixels having luminance values larger than the threshold value “ThL” are recognized as part of the retroreflective sheet 21 as imaged by evaluating small and large t of the luminance values on the basis of threshold value “ThL”.

In step S43, the high speed processor 91 increments the counter value “k” by one. In step S44, the high speed processor 91 determines whether or not the counter value “k” is “1”, and if k=1 the process proceeds to step S45, otherwise the process proceeds to step S46.

In step S45, the high speed processor 91 assigns the current Y-coordinate to the minimum Y-coordinate “minY”. In other words, after scanning starts from (X, Y)=(0, 0), “X” is incremented from “0” to “31” with “Y” which is fixed until X=31 but incremented each time “X” is returned to “0” and “X” is incremented again from “0” to “31” (refer to steps S55 to S59 to be described below), and thereby the value “Y” of the first array element “Dif [X] [Y]” (i.e., pixel) exceeding the threshold value “ThL” is necessarily the minimum Y-coordinate “minY”.

In step S46, the high speed processor 91 compares the current Y-coordinate with the current maximum Y-coordinate “maxY”. If the current Y-coordinate is larger than the current maximum Y-coordinate “maxY” in step S47, the high speed processor 91 proceeds to step S48, otherwise proceeds to step S49. In step S48, the high speed processor 91 assigns the current Y-coordinate to the maximum Y-coordinate “maxY”.

In step S49, the high speed processor 91 compares the current minimum X-coordinate minX and the current X-coordinate. If the current X-coordinate is smaller than the current minimum X-coordinate “minX” in step S50, the high speed processor 91 proceeds to step S51, otherwise proceeds to step S52. In step S51, the high speed processor 91 assigns the current X-coordinate to the minimum X-coordinate “minX”.

In step S52, the high speed processor 91 compares the current X-coordinate with the current maximum X-coordinate “maxX”. If the current X-coordinate is larger than the current maximum X-coordinate “maxX” in step S53, the high speed processor 91 proceeds to step S54, otherwise proceeds to step S55. In step S54, the high speed processor 91 assigns the current X-coordinate to the maximum X-coordinate “maxX”.

In step S55, the high speed processor 91 increments “X” by one. If X=32 in step S56 (i.e., when the process of one line of the differential image is finished), the high speed processor 91 proceeds to step S57, otherwise the high speed processor 91 proceeds to step S41.

In step S57, the high speed processor 91 assigns “0” to “X”. In step S58, the high speed processor 91 increments “Y” by one. Since one line of the differential image is completely processed, the steps S57 and S58 are taken to repeat the process for the next line.

If Y=32 in step S59 (i.e., when the process of the 32×32 pixels of the differential image is finished), the high speed processor 91 returns to the routine of FIG. 16, otherwise the high speed processor 91 proceeds to step S41.

The minimum X-coordinate “minX”, maximum X-coordinate “maxX”, minimum Y-coordinate “minY” and maximum Y-coordinate “maxY” are finally determined when Y=32 after repeating the above steps S41 to S59.

FIG. 18 is a flowchart showing an example of the process of determining two points in step S33 of FIG. 16. This flowchart is an example of the process of determining two points as explained with reference to FIG. 11.

As shown in FIG. 18, the high speed processor 91 assigns “0” to “M” in step S70, and repeatedly performs the process from step S71 to step S87. In this case, as shown in step S71, Ytb=minY in the first loop and Ytb=maxY in the second loop. In step S72, the high speed processor 91 starts scanning from the coordinates (minX, Ytb) as a starting point.

In step S73, the high speed processor 91 assigns “0” to the counter value “C1”. In step S74, the high speed processor 91 compares the differential data Dif [X] [Y] with the threshold value “ThL”, and proceeds to step S77 if the differential data is larger than the threshold value, otherwise proceeds to step S75. In step S75, the high speed processor 91 increments the count value “C1” by one. In step S76, the high speed processor 91 increments the coordinate “X” by one and proceeds to step S74.

In step S74, when it is determined that Dif [X] [Y]>ThL, the current counter value “C1” is equal to the distance “LT” or “LB” shown in FIG. 11 In step S72, C1=LT if Ytb=minY, and C1=LB if Ytb=maxY.

In step S77, the high speed processor 91 starts scanning from the coordinates (maxX, Ytb) as a starting point. In step S78, the high speed processor 91 assigns “0” to the counter value “Cr”. In step S79, the high speed processor 91 compares the differential data Dif [X] [Y] with the threshold value “ThL”, and proceeds to step S82 if the differential data is larger than the threshold value, otherwise proceeds to step S80. In step S80, the high speed processor 91 increments the count value “Cr” by one. In step S81, the high speed processor 91 decrements the coordinate “X” by one, and proceeds to step S79.

In step S79, when it is determined that Dif [X] [Y]>ThL, the counter value “Cr” is equal to the distance “RT” or “RB” shown in FIG. 11. In step S72, Cr=RT if Ytb=minY, and Cr=RB if Ytb=maxY.

In step 82, the high speed processor 91 compares the distance “Cr” with the distance “C1”. If the distance “C1” is larger than the distance “Cr” in step S83, the process proceeds to step S85, otherwise proceeds to step S84.

In step S84, “minX” is assigned to “Xtp[M]”, and “Ytb” is assigned to “Ytp[M]”. On the other hand, in step S85, “maxX” is assigned to “Xtp[M]”, and “Ytb” is assigned to “Ytp[M]”.

In this case, the coordinates (Xtp[0], Ytp[0]) are the coordinates of the first extraction point as explained with reference to FIG. 11, and the coordinates (Xtp[1], Ytp[1]) are the coordinates of the second extraction point as explained with reference to FIG. 11.

In step S86, the high speed processor 91 increments “M” by one, and proceeds to step S87. When the loop from step S71 to step S87 is finished, the process returns to the routine of FIG. 16.

FIG. 19 is a flowchart showing an example of the selection process in step S5 of FIG. 14 FIG. 19 is a flow chart showing an example of the process flow as explained with reference to FIGS. 8A and 8B and FIG. 9. As shown in FIG. 19, in step S101, the high speed processor 91 calculates the center coordinates between the center coordinates currently obtained and the center coordinates previously obtained, as calculated in step S34 of FIG. 16, and sets the current coordinates as calculated here to the current coordinates of the cursor 201 (called as the cursor coordinates). Incidentally, the center coordinates as used are screen coordinates after conversion. In step S102, if the current cursor coordinates are located in the selection acceptable area 211 in which the selection button 203U or 203D is included, the high speed processor 91 proceeds to step S103, otherwise proceeds to step S110.

In step S103, the high speed processor 91 sets an area flag to a value corresponding to the selection button 203U or 203D in the selection acceptable area 211 in which the current cursor coordinates are located. In step S104, the high speed processor 91 resets the current position of the cursor 201 to the center of the selection button 203U or 203D in the selection acceptable area 211 in which the current cursor coordinates are located. In step S105, the high speed processor 91 performs the settings of the animation of the indicator 202U or 202D which indicates the passage of time and is provided in the selection acceptable area 211 in which the current cursor coordinates are located. By this process, the corresponding indicator 202U or 202D is gradually filled with the predetermined color as time passes.

In step S106, the high speed processor 91 checks the area flag, determines whether or not its value is the same as the previous value, and if it is the same the process proceeds to step S108, otherwise proceeds to step S107. In step S107, the high speed processor 91 resets the elapsed time (returns it to 0), and proceeds to step S108. In step S108, the high speed processor 91 determines whether or not a predetermined time elapses, and if it elapses, the process proceeds to step S109, otherwise returns to the main routine of FIG. 14. In step S109, the high speed processor 91 performs the settings of animation and the display positions in order to display the next game mode selection screen in accordance with the direction of the arrow of the corresponding indicator 202U or 202D.

On the other hand, if the current cursor coordinates are located in the determination acceptable area 213 in step S110, the process proceeds to step S111, otherwise proceeds to step S118. In step S118, the high speed processor 91 sets the area flag to a value indicating that the current cursor coordinates are not located in either the selection button 203U or 203D and also not located in the OK button 207.

By the way, supplementary explanation will be given in regard to the process of FIG. 11, FIG. 17 and FIG. 18. FIG. 11 illustrates an exemplary case where both the glove type input articles 7L and 7R are imaged. However, even when only one of the glove type input articles 7L and 7R is imaged, it is apparent that, by the process of FIG. 17 and FIG. 18, the maximum X-coordinate “maxX”, minimum X-coordinate “minX”, maximum Y-coordinate “maxY” and minimum Y-coordinate “minY” can be obtained, and also the first extraction point and the second extraction point can be obtained.

On the other hand, in step S111, the high speed processor 91 sets the area flag to a value corresponding to the OK button 207 in the determination acceptable area 213 in which the current cursor coordinates are located. In step S112, the high speed processor 91 resets the current position of the cursor 201 to the center position of the OK button 207. In step S113, the high speed processor 91 sets the animation of the indicator 209 indicative of the passage of time. By this process, as time passes, the indicator 209 is gradually filled with a predetermined color in the clockwise direction.

In step S114, the high speed processor 91 checks the area flag, determines whether or not its value is the same as the previous value, and if it is the same the process proceeds to step S116, otherwise proceeds to step S115. In step S115, the high speed processor 91 resets the elapsed time (returns it to 0), and proceeds to step S116. In step S116, the high speed processor 91 determines whether or not a predetermined time elapses, and if it elapses, the process proceeds to step S117, otherwise returns to the main routine of FIG. 14. In step S117, the high speed processor 91 starts the process which is to be performed in the selected game mode. In this case, the game state is set to “fighting”.

FIG. 20 is a flowchart showing an example of the process flow during fighting in step S6 of FIG. 14. As shown in FIG. 20, in step S120, the high speed processor 91 determines which of the first extraction point and the second extraction point as obtained in the process of FIG. 18 is corresponding to the right or left hand.

In step S121, the high speed processor 91 evaluates the motion of the gloves 217L and 217R of the player's boxer, and determines a straight punch, a cross punch or no punch. In step S122, the high speed processor 91 updates the display position of the gloves 217L and 217R of the player's boxer. In step S123, the high speed processor 91 calculates the difference between the horizontal component (x component) of the center coordinates as currently obtained in step S34 of FIG. 16 and the horizontal component (x component) of the center coordinates as previously obtained, i.e., the moving distance of the center point in the horizontal direction, and adds it to an accumulated value “Dm” as previously obtained. As thus described, in step S123, the accumulated value “Dm” (i.e., sum of displacement “Dm”) is acquired by successively accumulating the moving distance of the center point in the horizontal direction, i.e., the moving distance of the center point between the glove 217L and the glove 217R in the horizontal direction. Incidentally, the center coordinates are screen coordinates after conversion.

In step S124, the high speed processor 91 controls the motion of the opposing boxer (i.e., the CPU boxer 215). In other words, the high speed processor 91 performs the settings of animation and position of the opposing boxer in accordance with the behavioral algorithm of the opposing boxer. In step S125, the high speed processor 91 controls the display of the background image in response to the position of the opposing boxer and the positions of the gloves 217L and 217R of the player's boxer.

More specific description is as follows. In the case of the present embodiment, horizontal 256 pixels (width)×vertical 224 pixels (height) are displayed in the screen of the television monitor 5, and three areas are defined by dividing the displayed area by three in the horizontal (width) direction. The background image is scrolled to the right direction when the opposing boxer moves from the center area to the left area, while the background image is scrolled to the left direction when the opposing boxer moves from the center area to the right area, in order to control the background image such that the opposing boxer is located in the center area. Such background control is equivalent to changing the angle of a camera when taking the image of the ring.

Also, the positions of the opposing boxer and the background image are controlled in accordance with the motions of the center point between the first extraction point and the second extraction point which are obtained by the process of FIG. 16. In other words, the positions of the opposing boxer and the background image are controlled in accordance with the moving distance of the center point between the first extraction point and the second extraction point in the horizontal direction. The opposing boxer and the background image are scrolled to the right when the center point moves to the left in relation to the center of the screen, and the opposing boxer and the background image are scrolled to the left when the center point moves to the right in relation to the center of the screen. Such control of the opposing boxer and the background image is performed for taking the motion parallax of the player's boxer into consideration.

In step S126, the high speed processor 91 determines whether or not a punch of the player's boxer hits the opposing boxer. In step S127, the high speed processor 91 determines whether or not a punch of the opposing boxer hits the player's boxer. More specifically speaking, if either the right or left punch of the opposing boxer is located between the glove 217L and the glove 217R of the player's boxer, it is determined that the punch hits the player's boxer. On the other hand, if either the right or left punch of the opposing boxer is located outside of the glove 217L or the glove 217R of the player's boxer, it is determined that the punch is defended.

In step S128, the high speed processor 91 determines whether or not the round ends, and if the round ends the process proceeds to step S129 to set the game state to “end of round”, and returns to the main routine, otherwise proceeds to step S130. In step S130, the high speed processor 91 whether or not the bout is over, i.e., all the rounds end, and if the bout is over, the process proceeds to step S131 to set the game state to “end of bout”, and returns to the main routine. On the other hand, if the bout is not over, the process returns to the main routine.

FIG. 21 is a flowchart showing an example of the right/left determination process in step S120 of FIG. 20. This flowchart is also an example of the process of determining right/left as explained with reference to FIG. 12. Incidentally, the position of the glove type input article 7L is called the left extraction point, and the position of the glove type input article 7R is called the right extraction point.

As shown in FIG. 21, in step S140, the high speed processor 91 predicts the current position (Xnl, Ynl) of the left extraction point from the previous position (XL[0], YL[0]) of the left extraction point. In step S141, the high speed processor 91 predicts the current position (Xnr, Ynr) of the right extraction point from the previous position (XR[0], YR[0]) of the right extraction point. In this case, the position (Xnl, Ynl) of the left extraction point corresponds to the predicted position TPLp of FIG. 12, and the position (Xnr, Ynr) of the right extraction point corresponds to the predicted position TPRp of FIG. 12.

In step S142, the high speed processor 91 assigns “0” to “M”. In step S143, the high speed processor 91 calculates the distance Dl between the predicted position (Xnl, Ynl) and the extraction point (Xtp[M], Ytp[M]). In step S144, the high speed processor 91 calculates the distance Dr between the predicted position (Xnr, Ynr) and the extraction point (Xtp [M], Ytp [M]).

In this case, the extraction point (Xtp[0], Ytp[0]) is the first extraction point as obtained by the routine of FIG. 18, and the extraction point (Xtp[1], Ytp[1]) is the second extraction point as obtained by the routine of FIG. 18. When M=0, the distance Dl corresponds to the distance LD1 of FIG. 12, and the distance Dr corresponds to the distance RD1 of FIG. 12. Also, when M=1, the distance D1 corresponds to the distance LD2 of FIG. 12, and the distance Dr corresponds to the distance RD2 of FIG. 12.

In step S145, the high speed processor 91 compares the distance Dr and the distance Dl. If Dl>Dr in step S146, the high speed processor proceeds to step S148 otherwise proceeds to step S147.

In step S147, the high speed processor 91 sets the position (XL[2], YL[2]) to the position (XL[1], YL[1]) of the left extraction point as determined twice before, and sets the position (XL[1], YL[1]) to the position (XL[0], YL[0]) of the left extraction point as determined just before. Then, the high speed processor 91 sets the coordinates (Xtp[M], Ytp[M]) to the current position (XL[0], YL[0]) of the left extraction point.

On the other hand, in step S148, the high speed processor 91 sets the position (XR[2], YR[2]) to the position (XR[1], YR[1]) of the right extraction point as determined twice before, and sets the position (XR[1], YR[1]) to the position (XR[0], YR[0]) of the right extraction point as determined just before. Then, the high speed processor 91 sets the coordinates (Xtp[M], Ytp[M]) to the current position (XR[0], YR[0]) of the right extraction point.

In step S149, the high speed processor 91 increments the variable “M” by one. In step S150, the high speed processor 91 determines whether or not M=2, and if M=2 the process returns to the routine of FIG. 20 otherwise proceeds to step S143.

FIG. 22 is a flowchart showing an example of the globe motion determination process in step S121 of FIG. 20. This flowchart is also an example of the process of determining the globe motion as explained with reference to FIG. 13.

As shown in FIG. 22, the high speed processor 91 repeats the process from step S160 to step S169. When i=0, the motion of the glove type input article 7L is determined, and when i=1, the motion of the glove type input article 7R is determined.

In step S161, the high speed processor 91 calculates the velocity vector Vi by the following equation.

Vi=(Xi[0]−Xi[2],Yi[0]−Yi[2])

In this case, the coordinates (X0[0], Y0[0]) are the current left extraction point (XL[0], YL[0]) corresponding to the left position TPL3 of FIG. 13, and the coordinates (X0[2], Y0[2]) are the left extraction point (XL[2], YL[2]) as determined twice before corresponding to the left position TPL1 of FIG. 13. Accordingly, the velocity vector VO corresponds to the velocity vector V of FIG. 13. On the other hand, the coordinates (X1[0], Y1[0]) are the current right extraction point (XR[0], YR[0]), and the coordinates (X1[2], Y1[2]) are the right extraction point (XR[2], YR[2]) as determined twice before.

In step S162, the high speed processor 91 places the start point of the velocity vector Vi at the origin in the virtual screen, and determines in which area the end point of the velocity vector Vi is located (refer to FIGS. 13B, 13D and 13F). If the end point of the velocity vector Vi is located in the “immovable area” in step S163, the high speed processor 91 proceeds to step S164 in which a immovable flag IFi is turned on, otherwise proceeds to step S165.

If the end point of the velocity vector Vi is located in the “straight area” in step S165, the high speed processor 91 proceeds to step S167 in which a straight flag SFi is turned on, otherwise the end point is located in the “cross area” and thereby the process proceeds to step S166 in which a cross flag CFi is turned on. In step S168, the high speed processor 91 increments the count value “Np” of a punch counter which counts the number of punches and the process proceeds to step S169. In this manner, the number of punches is counted with no distinction between straight and cross and between right and left.

After performing the process from step S160 to S169 twice, i.e., the determination process is completed for the left and right glove type input articles 7L and 7R, the process returns to the routine of FIG. 20.

FIG. 23 is a flowchart showing an example of the process of updating the positions of the gloves of the player's boxer in step S122 of FIG. 20. As shown in FIG. 23, the high speed processor 91 repeats the process from step S180 to S190. When i=0, the process is performed for the glove 217L of the player's boxer, and when i=1, the process is performed for the glove 217R of the player's boxer.

The high speed processor 91 determines in step S181 whether or not the immovable flag IFi is turned on, and if turned on, the process proceeds to step S182 in which the position of the corresponding glove 217L or 217R is updated, otherwise proceeds to step S184. In this case, the positions of the gloves 217L and 217R are set to the current position of the left extraction point and the current position of the right extraction point as converted in the screen coordinates respectively. However, the gloves 217L and 217R can be moved freely in the horizontal direction, but can be moved limitedly in the vertical direction (for example, the center of the gloves 217L and 217R can be moved only within the lower third of the screen). The globe 217L and 217R show an exemplary image (indicative of a basic figure) when there is no input from the player 11. In step S183, the high speed processor 91 turns off the immovable flag IFi.

The high speed processor 91 determines whether or not the straight flag SFi is turned on in step S184, and if turned on the process proceeds to step S185 in which settings are made to animate a straight punch with the corresponding glove 217L or 217R (an image indicative of an input by the player 11 (i.e., an image changing from basic figure), otherwise proceeds to step S187. In step S186, the high speed processor 91 turns off the straight flag SFi.

In step S187, the high speed processor 91 determines whether or not the cross flag CFi is turned on, and if turned on the process proceeds to step S188 in which settings are made to animate a cross punch with the corresponding glove 217L or 217R (an image indicative of another input by the player 11 (i.e., an image changing from the basic figure). In step S189, the high speed processor 91 turns off the cross flag CFi.

After performing the process from step S180 to S190 twice, i.e., after the updating process is completed for the left and right glove type input articles 7L and 7R, the process proceeds to step S191. In step S191, the high speed processor 91 determines whether or not the left globe 217L and the right globe 217R of the player's boxer are crossed, i.e., whether or not the relative positions thereof are switched between left and right, and if crossed the process proceeds to step S192. In step S192, the high speed processor 91 determines whether or not the left and right hands are continuous crossed for a predetermined time, and if this time elapses the process proceeds to step S193 in which settings are made to animate the globe 217L displayed at the right side and the globe 217R displayed at the left side which are switched each other left to right.

FIG. 24 is a flowchart showing an example of the calorie consumption calculation process in step S8 of FIG. 14. As shown in FIG. 24, in step S200, the high speed processor 91 divides the accumulated value “Dm” (i.e., sum of displacement “Dm”) of the moving distances of the center point between the glove 217L and the glove 217R in the horizontal direction, which is obtained in step S123 of FIG. 20, by “256” to acquire the quotient “Um”. In this case, the fractional residue is discarded. Here, while horizontal 256 pixels×vertical 224 pixels are displayed in the screen of the television monitor 5, the divisor “256” corresponds to the number of pixels in the horizontal direction. The 256 pixels are treated as one unit of displacement.

In step S201, the high speed processor 91 multiplies the quotient “Um” and a unit motion calorie consumption “Cm” (for example, 157 calories) to acquire the product “Ef”. In this case, the one unit, i.e., the unit motion calorie consumption “Cm” is the calorie consumption which is actually measured by having the player move the center point between the glove 217L and the glove 217R by 256 pixels in horizontal direction. Accordingly, the product “Ef” is the calorie consumption on the basis of the motion of the glove type input articles 7L and 7R in the horizontal direction.

In step S202, the high speed processor 91 multiplies a unit punch calorie consumption “Cp” (for example, 120 calories) by the count value “Np” as obtained in step S168 of FIG. 22, i.e., the number of punches “Np” to acquire the product “Es”. In this case, the unit punch calorie consumption “Cp” is the calorie consumption which is actually measured by having the player throw a punch. Thus, the product “Es” is the calorie consumption corresponding to the punches having been thrown. Incidentally, as understood from FIG. 22, the number of punches “Np” is counted with no distinction between straight and cross and between right and left.

In step S203, the high speed processor 91 acquires a calorie consumption “E (R)” of the current round (R+1) by adding the calorie consumption “Ef” of the horizontal motions of the glove type input articles 7L and 7R and the calorie consumption “Es” of the punches as thrown. The index R=0, 1, . . . , and the number (R+1) indicates the round number. In step S204, the high speed processor 91 clears the sum of displacement “Dm” and the number of punches “Np”.

Incidentally, the above actual measurements have been performed with Japanese women aged 20 in advance, and the unit motion calorie consumption “Cm” and the unit punch calorie consumption “Cp” calculated from the actual measurements are implemented as parameters of the calculation. In accordance with one preferred example of implementation, this unit motion calorie consumption “Cm” and this unit punch calorie consumption “Cp” are corrected by taking account of the age, gender and weight of the player which are entered by the player in order to obtain a value closer to the actual calorie consumption. In any cases, even if the calorie consumption as calculated includes some error, the amount of exercise by the player can be roughly recognized. In addition to this, since daily relative increase and decrease in the calorie consumption is substantially accurate, it is effective to display the calorie consumption for enabling the player's adherence to constant exercise and health maintenance. Returning to FIG. 14, in step S10, the high speed processor 91 sums up the calorie consumption “E (R)” as obtained in step S8 to calculate the total calorie consumption through the bout.

FIG. 25 is a view showing an exemplary screen in which the intermediate result is displayed on the basis of the processing result in step S9 of FIG. 14. As illustrated in FIG. 25, this screen contains a judgment result display area 500, a calorie consumption display area 502, an OK button 504 and the cursor 201. The judgments of the respective judges A to C of the current round (round 1 in the illustrated case) are displayed in the judgment result display area 500. In the figure, “Raz” is the name of the player's boxer.

Also, the calorie consumption (calculated in step S8) of the current round is displayed in the calorie consumption display area 502. Then, if the cursor 201 is located in the OK button 504 for a predetermined time, the process proceeds to the next round.

FIG. 26 is a view showing an exemplary screen in which the outcome of the fight is displayed on the basis of the processing result in step S11 of FIG. 14. As illustrated in FIG. 26, this screen contains a calorie consumption display area 506, a cancel button 510, an OK button 504, and the cursor 201. The total calorie consumption through the bout (as calculated in step S10) is displayed in the calorie consumption display area 506. In addition, the name of the winner is displayed (“Raz” in the figure).

Then, if the cursor 201 is located in the OK button 504 for a predetermined time, the calorie consumption, which is being displayed, is added to the accumulated value of the past calorie consumption. On the other hand, if the cursor 201 is located in the cancel button 510 for a predetermined time, the calorie consumption, which is being displayed, is not added to the accumulated value of the past calorie consumption.

FIG. 27 is a view showing an exemplary screen in which the total outcome is displayed after the outcome of the current fight is displayed in FIG. 26. As illustrated in FIG. 27, this screen contains a calorie consumption display area 506 and a total calorie consumption display area 508. The calorie consumption through the current bout (as calculated in step S10) is displayed in the calorie consumption display area 506. The accumulated value of the past calorie consumption is displayed in the total calorie consumption display area 508.

FIG. 28 is a view showing an exemplary screen in which comments are displayed after the total outcome is displayed in FIG. 27. As illustrated in FIG. 28, this screen contains a comment display area 514, a character 512, an OK button 504, and the cursor 201. Comments are displayed in the comment display area 514 in accordance with the outcome of the fight. Then, if the cursor 201 is located in the OK button 504 for a predetermined time, the process proceeds to step S5 of FIG. 14.

Incidentally, in accordance with the present invention as has been discussed above, the globe motion determination is performed on the basis of the position TPL3 as currently determined of the glove type input article 7L in the coordinates in which the past position TPL1 as determined twice before is located in the origin, (FIGS. 13A to 13F).

In other words, the origin is always located at the position determined by tracing back twice from the position to be currently determined, and thereby the motion determination is based on the relative position of the glove type input article 7L. Because of this, even if there are disparities in the body height of the player 11 and in the distance between the imaging unit 51 and the player 11, it is possible to display a constant glove image. This is true also for the glove type input article 7R.

In order to facilitate understanding of this feature, a motion determination process which is performed on the basis of the absolute position of the glove type input article 7L in the differential image will be considered. In this case, the differential image corresponds to the virtual screen. For example, when comparing a short player and a tall player playing with the glove type input article 7L in the same posture, needless to say, there is a difference between the positions of the glove type input article 7L gripped by the short and tall players in the differential image.

Accordingly, even if the short and tall players perform the similar action, the area where the glove type input article 7L of one is located may be different from the area where the glove type input article 7L of the other is located.

For example, while the glove type input article 7L is located in the straight area of the virtual screen when a tall player such as an adult throws a straight punch, the glove type input article 7L may be located in the immovable area of the virtual screen when a short player such as a child throws a straight punch. In such a case, although the similar action is taken, the glove image as displayed is different between a tall player and a short player. This shortcoming results also from the disparity in the distance between the imaging unit 51 and the player. It is not desirable that, in spite of the similar action, a different globe image is displayed depending upon the body height of the player or the distance between the imaging unit 51 and the player. This is true also for the glove type input article 7R. In accordance with the present embodiment, this shortcoming can be avoided.

Also, in the case of the present embodiment, there are the two virtual screens respectively for the two glove type input articles 7L and 7R, while the “straight area”, the “cross area” and the “immovable area” are defined for each of the glove type input articles 7L and 7R. Accordingly, a variety of glove images can be displayed respectively for the glove type input articles 7L and 7R in response to motions.

In order to facilitate understanding of this feature, it is assumed that only one virtual screen is provided for the two glove type input articles 7L and 7R. In such a case, a punch thrown with the glove type input article 7L is either a straight punch or a left cross punch (a punch toward the right), and a punch thrown with the glove type input article 7R is, either a straight punch or a right cross punch (a punch toward the left).

Accordingly, the glove type input article 7L when throwing a straight punch and the glove type input article 7R when throwing a right cross punch can be located in the same area. Needless to say, the opposite is true. In such a case, in spite of the different types of motions for left and right, the glove image corresponding to the glove type input article 7L and the glove image corresponding to the glove type input article 7R become similar, so that the glove image as displayed may not correspond to the actual motion by the player 11. For example, in the case where the glove type input article 7L when throwing a straight punch and the glove type input article 7R when throwing a right cross punch are located in the same “straight area” of the virtual screen, the same animation of a straight punch is displayed and therefore it is not appropriate as the glove image corresponding to the glove type input article 7R.

In this situation, eventually, glove images must be provided with no distinction between the types of punches with the glove type input articles 7L and 7R. Accordingly, it means nothing if the “straight area” and the “cross area” are distinctively defined. In other words, the respective motions of the glove type input articles 7L and 7R cannot be reflected in the glove images. In this regard, in accordance with the present embodiment, it is possible to display a variety of glove images (the animations of a straight punch and a cross punch) reflecting the motions of the glove type input articles 7L and 7R respectively.

Furthermore, in accordance with the present embodiment, when the current position TPL3 of the glove type input article 7L is located in the “immovable area” (refer to FIGS. 13A to 13F), the glove 217L is moved in the screen in synchronization with the glove type input article 7L (refer to FIG. 10). This is true also for the glove 217R. Accordingly, the player 11 can avoid and defend himself against a punch of the CPU boxer 215 by moving the glove type input articles 7L and 7R.

Furthermore, in accordance with the present embodiment, it is possible to display glove images reflecting the intention of the player 11. This point will be explained in detail. In accordance with the present embodiment, the glove image is displayed depending upon the area in which the current position TPL3 is located in the coordinates in which the position TPL1 of the glove type input article 7L as determined twice before is located in the origin. In this case, if the current position TPL3 is located in the “immovable area” including the origin, the image as displayed is indicative of the posture in which no punch is thrown (refer to the glove 217L of FIG. 10). Accordingly, when the motion of the glove type input article 7L is small, the current position TPL3 is often located in the “immovable area”, and thereby it is avoided as much as possible to determine, as a punch, a small motion of the player 11 which is not intended as a punch. This is true also for the glove type input article 7R.

Furthermore, the position TPL1 is used as the origin of the coordinates in which the globe motion determination is performed. Particularly, in this case, the position of the glove type input article 7L traced back twice from the current position TPL3 is used as the position TPL1. Because of this, in comparison with the case where the past position TPL2 which is determined once before is located at the origin, the displacement of the glove type input article 7L for a longer period can be used for determining the motion, and thereby when the glove type input article 7L is continuously moved, appropriate motion determination is possible along the motion thereof. Also, it is possible to enhance the difference between a small motion and a large motion of the glove type input article 7L. This is true also for the glove type input article 7R.

Furthermore, since the current positions of the glove type input articles 7L and 7R are determined on the basis of the currently predicted positions TPLp and TPRp of the glove type input articles 7L and 7R (refer to FIG. 12), even when the player 11 moves such that the glove type input article 7L and the glove type input article 7R are crossed to switch the relative positions thereof between left and right, the positions thereof can be determined correctly as much as possible (that is, left and right can be distinguished from each other).

Furthermore, in accordance with the present embodiment, since two points are extracted (i.e., the coordinates of the first and second extraction points are determined) on the assumption that both the glove type input articles 7L and 7R are imaged, it is possible to simplify the calculation for extracting the two points (refer to FIG. 11). This point will be explained in detail. If it is not assumed that both the two glove type input articles 7L and 7R are imaged, one shape or two shapes must be detected in the differential image. This is because it is possible both that both the two glove type input articles 7L and 7R are imaged and that only one article is imaged. Furthermore, it is required to calculate the center coordinates of one shape or two shapes as detected. Particularly, in the case where two shapes are located close to each other, it is difficult to determine which one or two glove type input article is imaged, and thereby the calculation of the center coordinates becomes quite difficult. In accordance with the present embodiment, since it is not necessary to perform the detection of the respective shapes and the calculation of the center coordinates, the above difficulties shall not rise and the calculation amount is small.

Furthermore, in accordance with the present embodiment, when the cursor coordinates are located in the area 211 or 213 including the button 203U, 203D or 207, the cursor 201 is moved to the center position of the button 203U, 203D or 207 irrespective of the positions of the glove type input articles 7L and 7R, so that the player 11 can easily move the cursor 201 to the button 203U, 203D or 207 only by bring the cursor 201 close to the button 203U, 203D or 207. In other words, when the cursor 201 is located close to the button 203U, 203D or 207, it is predicted that the player 11 intends to move the cursor 201 to the button 203U, 203D or 207, and thereby the cursor 201 is automatically moved to the button 203U, 203D or 207 for the purpose of lessening the operational burden of the player 11. In addition to this, since the elapsed time after the cursor 201 reaches the buttons 203U, 203D and 207 and the remaining time until a predetermined time elapses are displayed in the indicator 202U, 202D or 209, the player 11 can easily know the remaining time until the predetermined time at which the selection or determination is fixed, and thereby the user-friendliness for the player 11 can be improved (refer to FIGS. 8A and 8B, and FIG. 9).

Furthermore, in accordance with the present embodiment, since the adapter 1 into which the cartridge 3 is inserted is placed on the floor face for playing the boxing game, the displacements of the glove type input articles 7L and 7R in the differential image tend to be enlarged to reflect the motion of the player 11 appropriately. Meanwhile, even if the player 11 performs the same motion, when the adapter 1 with the cartridge 3 inserted thereinto is placed on the top surface of the television monitor 5, the glove type input articles 7L and 7R are moved in the differential image by smaller amounts than those when the adapter 1 is placed on the floor face.

Furthermore, in accordance with the preferred embodiment, the energy consumption of the player 11 can be easily calculated by the use of the result of stroboscopic imaging. In this case, it is possible to improve the accuracy of calculating energy consumption because the number of punches “Np” and the sum of displacement “Dm” are taken into consideration.

Embodiment 2

The hardware of the boxing game system of the embodiment 1 is used also as the hardware of the boxing game system of the embodiment 2. This boxing game system can perform an exercise process (mode) A, an exercise process (mode) B, an exercise process (mode) C and an exercise process (mode) D in addition to the boxing game process as has been discussed above in the description of the embodiment 1 with reference to FIG. 14. These processes will be explained in turn.

FIG. 29 is a view showing an example of an exercise screen displayed on the basis of the exercise process A performed by the boxing game system in accordance with the embodiment 2 of the present invention. As illustrated in FIG. 29, the high speed processor 91 displays ball objects 521A and 521B to appear one after another on the television monitor 5 in order that each object flies from the back side toward the front side. Also, the high speed processor 91 displays the glove 217L which moves in response to the motion of the glove type input article 7L and the glove 217R which moves in response to the motion of the glove type input article 7R on the television monitor 5.

The high speed processor 91 determines whether or not the ball object 521A is located in a predetermined range from the center position of the glove 217L or a predetermined range from the center position of the glove 217R, and if it is located in the predetermined range, the high speed processor 91 judges that the glove hits the ball object 521A. The high speed processor 91 counts the number of hits and moves (hits back) the ball object 521A that is hit in the backward direction. Incidentally, the motion control process of the gloves 217L and 217R and the process of calculating calorie consumption are performed in the same manner as those of the embodiment 1.

The player makes efforts to hit back the ball object 521A as much times as possible with the glove 217L or 217R by swinging the glove type input articles 7L and 7R. Meanwhile, if the ball object 521B is located in a predetermined range from the center position of the glove 217L or a predetermined range from the center position of the glove 217R, the ball object 521B is hit back, however, it is not judged as a hit so that the number of hits is not increased. In addition, the number of hits is displayed in the top left corner of the screen on a real time base, and the number of times that the ball object 521A appears is displayed in the top right corner of the screen on a real time base.

FIG. 30 is a view showing an example of an exercise screen displayed on the basis of the exercise process B performed by the boxing game system in accordance with the embodiment 2 of the present invention. As illustrated in FIG. 30, the high speed processor 91 displays an sandbag object 520 on the television monitor 5. Also, the high speed processor 91 displays the glove 217L which moves in response to the motion of the glove type input article 7L and the glove 217R which moves in response to the motion of the glove type input article 7R on the television monitor 5. Incidentally, the motion control process of the gloves 217L and 217R and the process of calculating calorie consumption are performed in the same manner as those of the embodiment 1.

The high speed processor 91 counts the number of punches which are thrown in a predetermined time (refer to step S168 of FIG. 22). The player makes efforts to throw punches as much times as possible with the glove 217L or 217R by swinging the glove type input articles 7L and 7R. Also, the number of punches is displayed in the top left corner of the screen on a real time base, and the elapsed time is displayed in the top right corner of the screen on a real time base.

FIG. 31 is a view showing an example of an exercise screen displayed on the basis of the exercise process C performed by the boxing game system in accordance with the embodiment 2 of the present invention. FIG. 32 is a view showing another example of the exercise screen of FIG. 31. As illustrated in FIG. 31, the high speed processor 91 displays panel objects P11, P12, P13, P21, P23, P31, P32 and P33, an opaque globe set 522, an instruction object 524, a guide 526 and a guide 528 on the television monitor 5.

The guide 526 contains eight rectangular figures corresponding respectively to the eight panel objects. The display position of the instruction object 524 is suggested by coloring (hatching in the exemplary illustration) some of the rectangular figures with a predetermined color. The arrow-shaped guide 528 shows the order of displaying the instruction object 524. The player can get the next position of the instruction object 524 by referring to the guides 526 and 528.

If the two areas 251 and 253 having higher luminance values as illustrated in FIG. 11 cannot be separately detected, but only one area having a higher luminance value is detected, then the high speed processor 91 displays the opaque globe set 522. On the other hand, if the two areas 251 and 253 having higher luminance values can be separately detected as illustrated in FIG. 11, then the high speed processor 91 displays a semi-transparent globe 530 as shown in FIG. 32. Accordingly, the player controls the location of the opaque globe set 522 by moving the glove type input articles 7L and 7R which are kept to be in contact with each other.

The high speed processor 91 displays the instruction object 524 overlapping the panel object in accordance with a program. If the glove set 522 is moved to the position of the instruction object 524 in response to the motion of the player, the high speed processor 91 increments a counter by one and display the instruction object 524 to overlap the panel object located in the position as suggested by the guide 526. The high speed processor 91 repeats such a process for a predetermined time.

The player moves the glove type input articles 7L and 7R in order to successively place the opaque globe set 522 over the instruction object 524 which is successively moving from one position to another. In addition, the number of times that the glove set 522 overlaps the instruction object 524 is displayed in the top left corner of the screen on a real time base, and the elapsed time is displayed in the top right corner of the screen on a real time base.

In this exercise process C, the glove set 522 is displayed in the position corresponding to the center point between the first extraction point and the second extraction point. Also, the calorie consumption is calculated on the basis of the accumulated amount of displacement of the glove set 522 in the horizontal direction and in the vertical direction. In this case, for each of the horizontal direction and the vertical direction, the process in step S123 of FIG. 20 and the process shown in FIG. 24 are performed, and the calorie consumption corresponding to the accumulated amount of displacement in the horizontal direction and the calorie consumption corresponding to the accumulated amount of displacement in the vertical direction are added.

FIG. 33 is a view showing an example of an exercise screen displayed on the basis of the exercise process D performed by the boxing game system in accordance with the embodiment 2 of the present invention. As illustrated in FIG. 33, the high speed processor 91 displays guides 534, 536, 538, 540 and 542, a target object 532 and the globes 217L and 217R on the television monitor 5. Incidentally, the motion control process of the gloves 217L and 217R and the process of calculating calorie consumption are performed in the same manner as those of the embodiment 1.

These guides instruct the player to throw a punch. In the example as illustrated, the guides 534, 536 and 540 instruct the player to throw a left straight punch, the guide 538 instructs the player to throw a right straight punch, and the guide 542 instructs the player to throw a right cross punch. Also, there is a guide prepared to instruct the player to throw a left cross punch.

A timing object 544 is displayed in order to surround the guide. Furthermore, an indicator 546 which grows along the edge of the timing object 544 as time passes is displayed. The player has to perform the action as suggested by the guide surrounded with the timing object 544 within the time indicated by the indicator 546 (before the indicator 546 has grown all around the guide).

The high speed processor 91 counts the number of times that the player performs the action as suggested by the guide surrounded with the timing object 544 within the time indicated by the indicator 546. Also, every time the leading end of the indicator 546 goes around the last guide in the screen (at the rightmost position) when the timing object 544 moves to this last guide, the high speed processor 91 switches the display of the guides and then the timing object 536 moves again from the first guide in the screen (at the leftmost position). In addition, the above number of times is displayed in the top left corner of the screen on a real time base, and the number of guides which have been displayed after starting this play in the top right corner of the screen on a real time base.

FIG. 34 is a schematic diagram showing the process transition among the routines performed by the boxing game system in accordance with the embodiment 2 of the present invention. As illustrated in FIG. 34, the high speed processor 91 displays a title (for example, “power boxing”) on the television monitor 5 in step S500. In step S501, the high speed processor 91 displays a save slot selection screen on the television monitor 5, and performs the process of selecting a save slot.

FIG. 35 is a view showing an example of the save slot selection screen displayed in step S501 of FIG. 34. As illustrated in FIG. 35, the high speed processor 91 displays the save slot 552, a selected slot indication area 554, screen changing objects 550L and 550R, a cancel button 510, an OK button 504 and the cursor 201 on the television monitor 5.

The save slot 552 of this example includes an upper section in which is displayed the current class and the current stage of the user in a championship mode, and an intermediate section in which is displayed the levels that are passed in the respective exercise modes A through D with star marks. Each of the exercise modes A through D is provided with 10 levels, and the player can start exercise from any level. Also, the save slot 552 includes a lower section in which is displayed the total calorie consumption. This total calorie consumption indicates the sum of all the calories consumed in the championship mode and the exercise mode.

There are four instances of the save slot 552 (i.e., for four users) having different colors from each other. For example, the colors may be red, blue, yellow and green. The user can easily confirm his own save slot 552 by the color.

The selected slot indication area 554 shows which instance of the save slot 552 is currently selected (displayed). Accordingly, when the player moves the cursor 201 to the screen changing object 550L or 550R by moving the glove type input articles 7L and 7R, another instance of the save slot 552 is displayed. The four instances of the save slot 552 are cyclically displayed by this operation. If the cursor 201 is located in the cancel button 510 for a predetermined time, the process proceeds to step S500, and if the cursor 201 is located in the OK button 504 for a predetermined time, the process proceeds to step S502.

Returning to FIG. 34, in step S502, the high speed processor 91 displays a play mode selection screen on the television monitor 5, and performs the process of selecting a play mode. This process is provided for selecting one of the championship mode, the exercise mode and a data view mode. When the player selects the championship mode, the high speed processor 91 proceeds to step S503; when the player selects the exercise mode, the high speed processor 91 proceeds to step S513; and when the player selects the data view mode, the high speed processor 91 proceeds to step S518. Incidentally, the player can select a mode by manipulating the cursor 201.

In step S503, the high speed processor 91 displays the mode selection screen and starts the process of selecting a mode. This process is provided for selecting one of a fighting mode and a training mode in the championship mode. When the player selects the fighting mode, the high speed processor 91 proceeds to step S504; and when the player selects the training mode, the high speed processor 91 proceeds to step S509. Incidentally, the player can select a mode by manipulating the cursor 201. The process of FIG. 14 is performed also in the fighting mode.

In step S504, the high speed processor 91 displays an opponent selection screen and starts the process of selecting an opponent. Incidentally, the player can select an opponent by manipulating the cursor 201. In step S505, the high speed processor 91 displays a class/stage selection screen, and starts the process of selecting a class and a stage. Incidentally, the player can select a class and a stage by manipulating the cursor 201.

In step S506, the high speed processor 91 performs the fighting process between the CPU boxer and the player's boxer. In step S507, the high speed processor 91 displays the outcome display screen on the television monitor 5 (refer to FIG. 26 and FIG. 27). In step S508, the high speed processor 91 displays a comment screen in accordance with the outcome of fighting (refer to FIG. 28).

On the other hand, in step S509, the high speed processor 91 displays the training selection screen, and starts the process of selecting a training mode. There are four training modes A to D from which the player can select one training mode by manipulating the cursor 201. The training modes A to D correspond respectively to the exercise modes A to D, and the processes thereof also correspond respectively to the exercise modes A to D as explained above (refer to FIG. 29 to FIG. 33 respectively). However, the player cannot select an arbitrary level in the training mode, but has to pass the respective levels in order.

In step S510, the high speed processor 91 performs the training mode as selected. In step S511, the high speed processor 91 displays the outcome display screen on the television monitor 5 (in the same manner as illustrated in FIG. 26 and FIG. 27). In step S512, the high speed processor 91 displays a comment screen (the screen as shown in FIG. 28) in accordance with the outcome in the training mode.

On the other hand, the high speed processor 91 displays an exercise selection screen in step S513, and starts the selection process in the exercise mode. There are the four exercise modes A to D (refer to FIG. 29 to FIG. 33) from which the player can select one exercise mode by manipulating the cursor 201.

FIG. 36 is a view showing an example of the exercise selection screen displayed in step S513 of FIG. 34. As illustrated in FIG. 36, the high speed processor 91 displays a selected exercise indication area 555, a passed level indication area 560, screen changing objects 550L and 550R, a cancel button 510, an OK button 504 and the cursor 201 on the television monitor 5.

The selected exercise indication area 555 shows which of the exercise mode is currently selected (displayed). More specific description is as follows. The selected exercise indication area 555 comprises four rectangular objects which are arranged in the horizontal direction. Each rectangular object is provided with an exercise name corresponding thereto. The rectangular object corresponding to the exercise mode currently selected is indicated by coloring (hatching in the exemplary illustration) it with a predetermined color. In addition, the name of the exercise mode currently selected (“Punch the red balls” in the illustrated example) is displayed in the center of the screen.

In the passed level indication area 560, the levels passed in the exercise mode currently selected are indicated by star marks. Each of the exercise modes A through D is provided with 10 levels, and the player can start exercise from any level.

When the player moves the cursor 201 to the screen changing object 550L or 550R by moving the glove type input articles 7L and 7R, the name of another exercise mode and the passed level indication area 560 are displayed, and the rectangle object of the selected exercise indication area 555 corresponding to the exercise mode as selected is colored with the predetermined color. If the cursor 201 is located in the cancel button 510 for a predetermined time, the process proceeds to step S502, and if the cursor 201 is located in the OK button 504 for a predetermined time, the process proceeds to step S514.

Returning to FIG. 34, in step S514, the high speed processor 91 displays a level selection screen, and performs the process of selecting a level. The player can select a level by manipulating the cursor 201.

FIG. 37 is a view showing an example of the level selection screen displayed in step S514 of FIG. 34. As illustrated in FIG. 37, the high speed processor 91 displays a level display area 561, screen changing objects 550L and 550R, a cancel button 510, an OK button 504 and the cursor 201 on the television monitor 5.

The level display area 561 is used to display the level as currently selected together with the requirements for passing the level and the predicted calorie consumption. When the player moves the cursor 201 to the screen changing object 550L or 550R by moving the glove type input articles 7L and 7R, the level display area 561 of another level is displayed. If the cursor 201 is located in the cancel button 510 for a predetermined time, the process proceeds to step S513, and if the cursor 201 is located in the OK button 504 for a predetermined time, the process proceeds to step S515.

Returning to FIG. 34, in step S515, the high speed processor 91 performs the process in the exercise mode as selected. In step S516, the high speed processor 91 displays the outcome display screens on the television monitor (the screen as shown in FIG. 26 and FIG. 27). In step S517, the high speed processor 91 displays the comment screen (the screen as shown in FIG. 28) in accordance with the outcome in the exercise mode.

On the other hand, the high speed processor 91 displays the contents of saved data on the television monitor 5 in step S518.

FIG. 38 is a view showing an example of the contents of saved data displayed in step S518 of FIG. 34. As illustrated in FIG. 38, the high speed processor 91 displays a first display area 572, a second display area 574, a third display area 576, screen changing objects 550L and 550R, an exit button 562, a data clear button 564 and the cursor 201 on the television monitor 5.

This screen is a screen displaying the contents of saved data in the championship mode. The first display area 572 is used to display a class, a stage and wins and losses. The second display area 574 is used to display the skills of the player's boxer by star marks. The skills includes four properties of power, speed, stamina and guard. The skill of speed is enhanced by passing the training mode corresponding to the exercise mode A; the skill of power is enhanced by passing the training mode corresponding to the exercise mode B; the skill of guard is enhanced by passing the training mode corresponding to the exercise mode C; and the skill of stamina is enhanced by passing the training mode corresponding to the exercise mode D. The third display area 576 is used to display the number of levels which are passed in the training mode and the number of levels which are failed in the training mode.

If the cursor 201 is located in the exit button 562 for a predetermined time, the process proceeds to step S502, and if the cursor 201 is located in the data clear button 564 for a predetermined time, the data displayed in the first display area 572, the second display area 574 and the third display area 576 is erased from an EEPROM. This EEPROM is not shown in the figures, but incorporated within the cartridge 3. If the cursor 201 is located in the screen changing objects 550L and 550R for a predetermined time, the screen is switched to the view shown in FIG. 39 or FIG. 40.

FIG. 39 is a view showing another example of the contents of saved data displayed in step S518 of FIG. 34. As illustrated in FIG. 39, the high speed processor 91 displays a passed level display area 570, screen changing objects 550L and 550R, an exit button 562, a data clear button 564 and the cursor 201 on the television monitor 5.

The passed level display area 570 is used to display the levels passed in the respective exercise modes A to D by star marks. If the cursor 201 is located in the exit button 562 for a predetermined time, the process proceeds to step S502, and if the cursor 201 is located in the data clear button 564 for a predetermined time, the data displayed in the passed level display area 570 is erased from the above EEPROM. If the cursor 201 is located in the screen changing objects 550L and 550R for a predetermined time, the screen is switched to the view shown in FIG. 38 or FIG. 40.

FIG. 40 is a view showing a further example of the contents of saved data displayed in step S518 of FIG. 34. As illustrated in FIG. 40, the high speed processor 91 displays a calorie consumption display area 566, a punch number display area 568, screen changing objects 550L and 550R, an exit button 562, a data clear button 564 and the cursor 201 on the television monitor 5.

The calorie consumption display area 566 is used to display the total calorie consumption as accumulated in the championship mode, the total calorie consumption as accumulated in the exercise mode, and the sum thereof. The punch number display area 568 is used to display the total number of punches as accumulated in the championship mode and the exercise mode.

If the cursor 201 is located in the exit button 562 for a predetermined time, the process proceeds to step S502, and if the cursor 201 is located in the data clear button 564 for a predetermined time, the data displayed in the calorie consumption display area 566 and the punch number display area 568 is erased from an EEPROM. If the cursor 201 is located in the screen changing objects 550L and 550R for a predetermined time, the screen is switched to the view shown in FIG. 38 or FIG. 39.

As has been discussed above, the present embodiment provides not only boxing bouts, but also a variety of exercise modes. Accordingly, the player can enjoy not only a game but also exercise. In addition to this, since the calorie consumption is displayed in either the championship mode or the exercise mode, the player can quantitatively know how much calories are consumed.

Also, the above EEPROM stores the data as shown in the screens of FIG. 38 to FIG. 40. This data can be cleared if necessary, and therefore the user can save the data and know the course of the outcome each time the data is cleared.

Meanwhile, the present invention is not limited to the above embodiments, and a variety of variations and modifications may be effected without departing from the spirit and scope thereof, as described in the following exemplary modifications.

(1) While a cartridge type is employed in the above description, it is possible to implement the respective functions of the cartridge 3 within the adapter 1 without the use of a cartridge.

(2) In the above description, the globe motion determination process is performed by locating the past position which is determined twice before (FIGS. 13A to 13F) at the origin. However, the number of times that the past position is traced back is not limited to this, but can be set to three or more times if appropriate through a trial and error process. In addition, as shown in FIG. 14, one cycle of the process is completed before an interrupt is issued by the next video system synchronous signal. In other words, one cycle of the process is completed within one video frame. However, it is possible to complete one cycle of the process in N video frames (N is two or a larger integer) such as two video frames. For example, if one cycle of the process is completed in two video frames, the positions of the glove type input articles 7L and 7R are calculated once per two video frames.

(3) In the above right/left determination process, as shown in FIG. 12, the predicted position TPLp and TPRp of the glove type input articles 7L and 7R are calculated only on the basis of the velocity vectors VL and VR which are calculated from the positions TPL1 and TPR1 determined twice before and the previous positions TPL2 and TPR2. However, it is possible to calculate the predicted positions TPLp and TPRp by using also the positions TPL0 and TPR0 which are determined before the positions TPL1 and TPR1 determined twice before. The positions TPL0, TPL1 and TPL2 are considered (left prediction). A velocity vector VL0 is calculated such that the position TPL0 is the start point thereof and the position TPL1 is the end point thereof, and a velocity vector VL1 is calculated such that the position TPL1 is the start point thereof and the position TPL2 is the end point thereof. A predicted vector VLp is determined in order that the angle between the velocity vector VL1 and the predicted vector VLp is equal to the angle between the velocity vectors VL0 and VL1. Furthermore, the magnitude of the velocity vector VL1 is multiplied by the ratio “r”, which is calculated as r=(magnitude of the velocity vector VL1)/(magnitude of the velocity vector VL0), and the magnitude of the predicted vector VLp is set to the result of multiplication. Then, the starting point of the predicted vector VLp is set to the end point of the velocity vector VL1, and the predicted position TPLp is set to the end point of the predicted vector VLp. The right prediction is performed in the same manner. By this process, the predicted position can be calculated with a high degree of accuracy.

(4) In addition to the configuration of the above embodiment, it is possible to implement, within the respective glove type input articles 7L and 7R, an acceleration sensor circuit, an infrared light emitting diode, a microcomputer and so forth as described in Japanese Patent Published Application No. Hei 2004-49436. The microcomputer controls the acceleration sensor circuit, and receives acceleration information therefrom. Then, the microcomputer drives the infrared light emitting diode in order to transmit acceleration information of the glove type input articles 7L and 7R to the adapter 1 by infrared communication. Accordingly, the high speed processor 91 makes use of the acceleration information to determine whether or not a punch is thrown by moving the glove type input articles 7L and 7R, and makes use of the result of imaging by the imaging unit 51 to determine motions for avoiding or protecting against the punch of the opposing boxer. By this configuration, the adapter 1 with the cartridge 3 inserted thereinto can be placed on the television monitor 5 without causing any problem to play game.

(5) In the above description, the elapsed time and the remaining time are not represented by numerals but represented by variation in color as illustrated in FIG. 8 and FIG. 9. However, the representation method is not limited to this, but they can be represented by numerals, change in shape, or any other arbitrary method.

(6) In the above description, the virtual screen is divided into the “immovable area”, the “straight area” and the “cross area”. In addition, the gloves in the basic posture, a straight punch, or a cross punch are displayed in accordance with the area in which the input article 7L or 7R is located. However, the configuration of the virtual screen is not limited thereto, but it is possible to increase or decrease the number of areas, and change the actions assigned to the respective areas (what image is to be displayed or what process is to be performed when the input article is located in the area).

(7) In the above description, there are two virtual screens which are mirror images each other in the right and left direction. This is because the function (for punches) of the left-handed input article 7L is the same as the function (for punches) of the right-handed input article 7R. However, the virtual screens are not necessarily mirror images each other in the right and left direction, but left and right virtual screens which are totally different from each other can be used in accordance with the type of the game. For example, in the case where the function (for example, for moving a shield) of the left-handed input article is different from the function (for example, for swinging a sword) of the right-handed input article, left and right virtual screens which are different from each other may be used. That virtual screens are different means that they are different in the number, dimensions and/or functions of areas which are defined in the virtual screen.

(8) While the boxing game is explained in the above description, the application program is not limited to this and not limited to games. Also, depending upon the context of the application, it is possible to arbitrarily select the shapes of input articles and the locations of the retroreflective sheets attached to the input articles.

(9) In the above description, the amount of exercise by the player is represented by energy consumption in units of calories. However, the unit is not limited to calories, but any other unit of energy can be used. Also, while energy consumption is used to directly represent the amount of exercise by the player, any appropriate representation can be used to indirectly represent the amount of exercise by the player. For example, it may be suggested what number of apples are equivalent to the exercise, what number of steps are equivalent to the exercise and so forth. As has been discussed above, in this description, “the amount of exercise” is meant a value which quantitatively represents how much the player exercises.

(10) In the above description, the motion control of the gloves 217L and 217R and the calculation of calorie consumption are performed on the basis of the positional information of the glove type input articles 7L and 7R as the state information thereof. However, speed information, moving direction information, moving distance information, velocity vector information, acceleration information, movement locus information, area information (perspective information), and/or positional information can be calculated as state information of glove type input articles 7L and 7R in order to calculate energy consumption on basis of the state information.

(11) It is possible to calculate the calorie consumption by detecting arbitrary motions of the player 11 during playing game as shown in FIG. 10 and so on, and also by displaying images on the television monitor 5 through which the high speed processor 91 instructs the player 11 what motion to do as shown in FIG. 31 to FIG. 33 and detecting the motion that the player 11 actually performs. (12) By inserting a binarization step between step S31 and step S32 of FIG. 16, the differential image is converted into binary image data by comparing the threshold value ThL and the array element “Dif [X] [Y]”, and the processes of steps S32 and S33 are performed on the basis of the binary image data. In this case, if the array elements “Dif [X] [Y]” larger than the threshold value ThL are set to “1” and the array elements “Dif [X] [Y]” smaller than the threshold value ThL are set to “0”, then the threshold value ThL for use in the processes of FIG. 17 and FIG. 18 is replaced, for example, by a value of “0”.

(13) In accordance with the present invention, the player is informed of the amount of exercise he actually do in terms of calorie consumption to maintain the health of body. In view of this point, besides boxing, there are a variety of exercises to which the present invention can be applied. In any way, the player puts on some retroreflective portion before doing an exercise.

The foregoing description of the embodiments has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and obviously many modifications and variations are possible in light of the above teaching. The embodiment was chosen in order to explain most clearly the principles of the invention and its practical application thereby to enable others in the art to utilize most effectively the invention in various embodiments and with various modifications as are suited to the particular use contemplated. 

1. A boxing game processing method comprising: an illumination step of emitting infrared light in a predetermined cycle to illuminate a left-handed glove type input article and a right-handed glove type input article which are provided respectively with retroreflective surfaces; an image generation step of imaging the left-handed glove type input article and the right-handed glove type input article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a differential data generation step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a position calculation step of calculating positional information of the left-handed glove type input article and the right-handed glove type input article on the basis of the differential data; an area determination step of determining in which area the position of the left-handed glove type input article is located in a first virtual screen which is divided into a straight area, a cross area and an immovable area, wherein said position of the left-handed glove type input article is a relative position which is indicated by current positional information of the left-handed glove type input article and converted into a coordinate system, the origin of which is located in the position indicated by past positional information obtained by tracing back for a predetermined number of times; an area determination step of determining in which area the position of the right-handed glove type input article is located in a second virtual screen which is divided into a straight area, a cross area and an immovable area, wherein said position of the right-handed glove type input article is a relative position which is indicated by current positional information of the right-handed glove type input article and converted into a coordinate system, the origin of which is located in the position indicated by past positional information obtained by tracing back for a predetermined number of times; a display step of displaying a left glove image which represents the left-handed glove type input article and a right glove image which represents the right-handed glove type input article in accordance with the result of determination in said area determination steps for the left-handed glove type input article and the right-handed glove type input article, wherein the first virtual screen and the second virtual screen are provided as mirror images each other in the right and left direction, and wherein in said area determination step for the left-handed glove type input article, an image showing that a left straight punch is thrown is displayed as the left glove image when the relative position indicated by the current positional information of the left-handed glove type input article is located in the straight area which does not include the origin, an image showing that a left cross punch is thrown is displayed as the left glove image when the relative position indicated by the current positional information of the left-handed glove type input article is located in the cross area which does not include the origin, an image showing that no left punch is thrown is displayed as the left glove image when the relative position indicated by the current positional information of the left-handed glove type input article is located in the immovable area which includes the origin, in said area determination step for the right-handed glove type input article, an image showing that a right straight punch is thrown is displayed as the right glove image when the relative position indicated by the current positional information of the right-handed glove type input article is located in the straight area which does not include the origin, an image showing that a right cross punch is thrown is displayed as the right glove image when the relative position indicated by the current positional information of the right-handed glove type input article is located in the cross area which does not include the origin, an image showing that no right punch is thrown is displayed as the right glove image when the relative position indicated by the current positional information of the right-handed glove type input article is located in the immovable area which includes the origin.
 2. The boxing game processing method as claimed in claim 1 further comprising: a step of obtaining a first extraction point indicative of the position of the left-handed glove type input article or the left-handed glove type input article on the basis of the differential data; a step of obtaining a second extraction point indicative of the position of the left-handed glove type input article or the left-handed glove type input article on the basis of the differential data; a step of predicting the current position of the left-handed glove type input article on the basis of past positional information of the left-handed glove type input article; a step of predicting the current position of the right-handed glove type input article on the basis of past positional information of the right-handed glove type input article; a step of calculating a first distance which is a distance between the first extraction point and the current position as predicted of the left-handed glove type input article; a step of calculating a second distance which is a distance between the first extraction point and the current position as predicted of the right-handed glove type input article; a step of setting the current position of the right-handed glove type input article to the first extraction point if the first distance is larger than the second distance, and setting the current position of the left-handed glove type input article to the first extraction point if the second distance is larger than the first distance; a step of calculating a third distance which is a distance between the second extraction point and the current position as predicted of the left-handed glove type input article; a step of calculating a fourth distance which is a distance between the second extraction point and the current position as predicted of the right-handed glove type input article; a step of setting the current position of the right-handed glove type input article to the second extraction point if the third distance is larger than the fourth distance, and setting the current position of the left-handed glove type input article to the second extraction point if the fourth distance is larger than the third distance.
 3. The boxing game processing method as claimed in claim 2 further comprising: a step of obtaining a maximum horizontal coordinate of pixels that have luminance values larger than a predetermined threshold value in an image on the basis of the differential data; a step of obtaining a minimum horizontal coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a maximum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a minimum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data, wherein said step of obtaining the first extraction point comprising: a step of obtaining a first horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a second horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the first horizontal distance is larger than the second horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the second horizontal distance is larger than the first horizontal distance; said step of obtaining the second extraction point comprising: a step of obtaining a third horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a fourth horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the third horizontal distance is larger than the fourth horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the fourth horizontal distance is larger than the third horizontal distance;
 4. The boxing game processing method as claimed in claim 3 further comprising: a step of moving a cursor on a screen to follow the variation of the position of the left-handed glove type input article and/or the right-handed glove type input article; a step of displaying an input area on the screen for receiving an input from an operator; a step of moving the position of the cursor to a predetermined position in the input area if the cursor is located in a predetermined area including the input area irrespective of the positions of the left-handed glove type input article and the right-handed glove type input article; a step of displaying an image indicative of the elapsed time after the cursor is located in the predetermined position and/or the remaining time until a predetermined time elapses on the screen; and a step of performing a predetermined process when the cursor is located in the predetermined area at least in the predetermined time.
 5. A display control method comprising: an illumination step of emitting infrared light in a predetermined cycle to illuminate a plurality of input articles which are provided respectively with retroreflective portions an image generation step of imaging the plurality of input articles both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a differential data generation step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; and a position calculation step of calculating positional information of the plurality of input articles respectively in the image on the basis of the differential data, wherein a plurality of virtual screens are provided respectively corresponding to the plurality of input articles, said display control method further comprising: an area determination step of determining in which area the position of the input article is located in the corresponding virtual screen which is divided into a plurality of areas, wherein said position of the input article is a relative position which is indicated by current positional information of the input article and converted into a corresponding coordinate system, the origin of which is located in the position indicated by past positional information of the input article obtained by tracing back for a predetermined number of times, wherein an image corresponding to each of the plurality of input articles is displayed in accordance with the result of area determination of the each of the plurality of input articles by said area determination step.
 6. The display control method as claimed in claim 5 wherein the predetermined number of times is a plural number.
 7. The display control method as claimed in claim 5 wherein the virtual screen is divided into at least two areas including a first area and a second area, and wherein in said area determination step, an image is displayed as an image corresponding to the input article for showing that an input is made when the relative position indicated by the current positional information of the input article is located in the first area which does not include the origin, and an image is displayed as an image corresponding to the input article for showing that no input is made when the relative position indicated by the current positional information of the input article is located in the second area which includes the origin.
 8. The display control method as claimed in claim 5 wherein the virtual screen is divided into at least three areas including a first area, a second area and a third area, and wherein in said area determination step, an image is displayed as an image corresponding to the input article for showing that a first input is made when the relative position indicated by the current positional information of the input article is located in the first area which does not include the origin, an image which is different from the image showing that the first input is made is displayed as an image corresponding to the input article for showing that a second input is made when the relative position indicated by the current positional information of the input article is located in the second area which does not include the origin, and an image is displayed as an image corresponding to the input article for showing that no input is made when the relative position indicated by the current positional information of the input article is located in the third area which includes the origin.
 9. A position detection method comprising: a step of emitting infrared light in a predetermined cycle to illuminate a first input article and a second input article which are provided respectively with retroreflective portions a step of imaging the first input article and the second input article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a step of obtaining a first extraction point indicative of the position of the first input article or the second input article on the basis of the differential data; a step of obtaining a second extraction point indicative of the position of the first input article or the second input article on the basis of the differential data; a step of predicting the current position of the first input article on the basis of past positional information of the first input article; a step of predicting the current position of the second input article on the basis of past positional information of the second input article; a step of calculating a first distance which is a distance between the first extraction point and the current position as predicted of the first input article; a step of calculating a second distance which is a distance between the first extraction point and the current position as predicted of the second input article; a step of calculating a third distance which is a distance between the second extraction point and the current position as predicted of the first input article; a step of calculating a fourth distance which is a distance between the second extraction point and the current position as predicted of the second input article; a step of setting the current position of the second input article to the first extraction point if the first distance is larger than the second distance, and setting the current position of the first input article to the first extraction point if the second distance is larger than the first distance; a step of setting the current position of the second input article to the second extraction point if the third distance is larger than the fourth distance, and setting the current position of the first input article to the second extraction point if the fourth distance is larger than the third distance.
 10. The position detection method as claimed in claim 9 further comprising: a step of obtaining a maximum horizontal coordinate of pixels that have luminance values larger than a predetermined threshold value in an image on the basis of the differential data; a step of obtaining a minimum horizontal coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a maximum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a minimum vertical coordinate of pixels that have luminance values larger than the predetermined threshold value in the image on the basis of the differential data, wherein said step of obtaining the first extraction point comprising: a step of obtaining a first horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a second horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the minimum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the first horizontal distance is larger than the second horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the first extraction point and the minimum vertical coordinate to the vertical coordinate of the first extraction point if the second horizontal distance is larger than the first horizontal distance; said step of obtaining the second extraction point comprising: a step of obtaining a third horizontal distance which is a horizontal distance from a starting position of the minimum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of obtaining a fourth horizontal distance which is a horizontal distance from a starting position of the maximum horizontal coordinate and the maximum vertical coordinate to the position of the pixel having the luminance value of which first exceeds the predetermined threshold value in the image on the basis of the differential data; a step of setting the maximum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the third horizontal distance is larger than the fourth horizontal distance, and setting the minimum horizontal coordinate to the horizontal coordinate of the second extraction point and the maximum vertical coordinate to the vertical coordinate of the second extraction point if the fourth horizontal distance is larger than the third horizontal distance.
 11. A cursor control method comprising: a step of emitting infrared light in a predetermined cycle to illuminate an input article which is provided with a retroreflective portion; a step of imaging the input article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a step of calculating the position of the input article on the basis of the differential data; a step of moving a cursor on a screen to follow the variation of the position of the input article; a step of displaying an input area on the screen for receiving an input from an operator; a step of moving the position of the cursor to a predetermined position in the input area if the cursor is located in a predetermined area including the input area irrespective of the position of the input article; a step of displaying an image indicative of the elapsed time after the cursor is located in the predetermined position and/or the remaining time until a predetermined time elapses on the screen; and a step of performing a predetermined process when the cursor is located in the input area at least for the predetermined time.
 12. An energy consumption calculating method comprising: a step of emitting infrared light in a predetermined cycle to illuminate an operation article operated by a user; a step of imaging the operation article both when the infrared light is emitted and when the infrared light is not emitted, and generating image data obtained with illumination and image data obtained without illumination; a step of generating differential data between the image data obtained with illumination and the image data obtained without illumination; a step of calculating state information of the operation article the basis of the differential data; and a step of calculating energy consumption when the user operates the operation article on the basis of the state information.
 13. The energy consumption calculating method as claimed in claim 12 wherein said step of calculating said energy consumption comprising: a step of determining which of a plurality of motion patterns the motion of the operation article belongs; a step of calculating said energy consumption on the basis of the motion pattern as determined.
 14. The energy consumption calculating method as claimed in claim 12 wherein said step of calculating said energy consumption comprising: a step of calculating a moving distance of the operation article on the basis of positional information as the state information; and a step of calculating said energy consumption on the basis of the moving distance as calculated.
 15. The energy consumption calculating method as claimed in claim 12 wherein said step of calculating said energy consumption comprising: a step of determining which of a plurality of motion patterns the motion of the operation article belongs; a step of calculating first energy consumption on the basis of the motion pattern as determined; a step of calculating a moving distance of the operation article on the basis of positional information as the state information; a step of calculating second energy consumption on the basis of the moving distance as calculated; and a step of calculating said energy consumption by adding the first energy consumption and the second energy consumption.
 16. The energy consumption calculating method as claimed in claim 12 wherein the state information is one of or any combination of two or more of speed information, moving direction information, moving distance information, velocity vector information, acceleration information, movement locus information, area information, and positional information.
 17. An exercise system comprising: an infrared light emission unit operable to periodically emit infrared light to a retroreflective portion which an exerciser puts on; an infrared light image sensor operable to detect the infrared light as reflected by the retroreflective portion to obtain a series of image data a signal processing unit connected to said infrared light image sensor, and operable to generate a first image indicative of an exercise that the exerciser to do, receive the series of image data of the retroreflective portion from said infrared light image sensor while the exerciser does the exercise, calculate calorie consumption estimated of the exerciser, and generate a second image indicative of the calorie consumption, wherein the calorie consumption is calculated on the basis of the motion of the retroreflective portion corresponding to the exercise that the exerciser has done with reference to the series of image data obtained by said infrared light image sensor. 